-- Logs begin at Fri 2022-07-15 18:41:54 UTC, end at Fri 2022-07-15 20:15:43 UTC. -- Jul 15 19:10:36.733140 dv-cinder-849690-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jul 15 19:10:36 2022] *** Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: nodename: dv-cinder-849690-2-iscsi Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: machine: x86_64 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: clock source: unix Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: pcre jit disabled Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: detected number of CPU cores: 4 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: current working directory: / Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: detected binary path: /usr/bin/uwsgi-core Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: your processes number limit is 47680 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: your memory page size is 4096 bytes Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: detected max file descriptor number: 2048 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: lock engine: pthread robust mutexes Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: thunder lock: enabled Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 15 19:10:36.746190 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 15 19:10:36.784041 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: Python main interpreter initialized at 0x5618fd3a29c0 Jul 15 19:10:36.784041 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: python threads support enabled Jul 15 19:10:36.784246 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: your server socket listen backlog is limited to 100 connections Jul 15 19:10:36.784246 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: your mercy for graceful operations on workers is 90 seconds Jul 15 19:10:36.784384 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: mapped 403077 bytes (393 KB) for 2 cores Jul 15 19:10:36.784495 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: *** Operational MODE: preforking *** Jul 15 19:10:36.784770 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: *** uWSGI is running in multiple interpreter mode *** Jul 15 19:10:36.784770 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: spawned uWSGI master process (pid: 80192) Jul 15 19:10:36.784960 dv-cinder-849690-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 15 19:10:36.791006 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: spawned uWSGI worker 1 (pid: 80199, cores: 1) Jul 15 19:10:36.791006 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: spawned uWSGI worker 2 (pid: 80200, cores: 1) Jul 15 19:10:36.791006 dv-cinder-849690-2-iscsi devstack@keystone.service[80192]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 15 19:10:42.179074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:10:42.179074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: from cryptography.utils import int_from_bytes Jul 15 19:10:42.179872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:10:42.179872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: from cryptography.utils import int_from_bytes Jul 15 19:10:42.180010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:10:42.180010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: from cryptography.utils import int_from_bytes Jul 15 19:10:42.180471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 15 19:10:42.180471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: from cryptography.utils import int_from_bytes Jul 15 19:10:42.229809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 19:10:42.229992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 19:10:42.230181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] command line args: [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 19:10:42.230450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 19:10:42.230629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ================================================================================ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 19:10:42.230792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 15 19:10:42.230885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] admin_token = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.230992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 15 19:10:42.231119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config_dir = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.231205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] command line args: [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 15 19:10:42.231311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.231439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 15 19:10:42.231528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] config_source = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.231605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ================================================================================ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 15 19:10:42.231676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] debug = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.231849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] admin_token = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config_dir = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] config_source = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] debug = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.232967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_config_append = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_dir = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.233987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_config_append = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_dir = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.234986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.235881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.236903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] publish_errors = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.237996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] publish_errors = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.238982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] standard_threads = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.239888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] standard_threads = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_journal = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_json = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_stderr = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] use_syslog = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.240981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_journal = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.241849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.242298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_json = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.242425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.242554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_stderr = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.242671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.242786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] use_syslog = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.242901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 15 19:10:42.243147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.243993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.244967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.245916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.246935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.247887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.248148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.248273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.248390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.248390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.248604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.248968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.249918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.250999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.251925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.252995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.253919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.254951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.255873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.256004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.256141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.256262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.256378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.256493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.256597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.257362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.257853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.257980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.258953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.259981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.260896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.261926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.262979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.263938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.264925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.265954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.266911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.267969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.external = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.268961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.external = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.password = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.269982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] auth.token = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.password = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.270861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] auth.token = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.271913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.272028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.272162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.272347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.272514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.273989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.274948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.275925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.276915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.277980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.278930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.279978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.280950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.281892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.282976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.283926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.284908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.285886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.286951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.287919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.288510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.288661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.288781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.288898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.289977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.290922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.291964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.292943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.293881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.294946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.295896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.296986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.297935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.298878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.299960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.300924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.301988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.302812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.303197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.303315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.303431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.304569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.304699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.304818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.304934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.305882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.306959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.307906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.308874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.309995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.310950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.311896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.312987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.313931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.314990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.315943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.316948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.317897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.318984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.319135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.319948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.320418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.320538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.320655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.320771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.320903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.321984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.322951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80199) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 19:10:42.323186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.323186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 15 19:10:42.323186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80200) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 15 19:10:42.348243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.348378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.348473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.348551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.348692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.348777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.356981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.357167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.385917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.386139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.386226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.386422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.528942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.529355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.578099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.578329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.578545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.578781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.642144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.642363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.642585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.642668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.642804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.642888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.643020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.643102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.643233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.643315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.643451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.643671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.667975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.668964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.669089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.669171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.669302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.669384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.669521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.680201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.680567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.692760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.692987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.693942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.694072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80200) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.694675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.694823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80199) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 15 19:10:42.711660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.711866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:10:42.712229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.712393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.712475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:10:42.712605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:10:42.712914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.712995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.713124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 15 19:10:42.713204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.713423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.713627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.713707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.713895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.714159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.714422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.724502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.724778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.725044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.725301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.725882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.726280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.726636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.726921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.734194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.735220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.737532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.737805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.738082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.738361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.738549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.738659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.738827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.738936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.739915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.740025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.740199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.740307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.740471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.740743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.741075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.741406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.768605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.769419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.770230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.770501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.771052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.771336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.777056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.777333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.777589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.777671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.777870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.778154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.785150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.785385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.785647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.785927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.790032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.790318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.790545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.790628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.790832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.790914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.791921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.792003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.792220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.792491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.810841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.811113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.811434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.811711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.815664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.816299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.819941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.820255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.820803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.821096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.825414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.826270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.829093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.829943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.832745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.833040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.833334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.833599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.833682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.833893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.833976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.834173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.834255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.834458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.835327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.835624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.854913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.855212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.855480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.855745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.855849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.856040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.856328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.856594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.863725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.864554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.865472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.865748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.866896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.867165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.869516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.869795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.870076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.870373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.870459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.870655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.870743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.870949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.871238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.871522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.885000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.885283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.885566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.885847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.885930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.886125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.886205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.886414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.886494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.886687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.886767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.886967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.887255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.887524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.906232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.906942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.909526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.910249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.912923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.913662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.914546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.915273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.916354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.916633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.916912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.917079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.917354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.917631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.924253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.924936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.925867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.926546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.929804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.930094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.930337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.930419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.930622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.930715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.930902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.931181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.939903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.940212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.940495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.940588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.940777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.941052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.948993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80199) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.949272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.949529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80200) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 15 19:10:42.949612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.949806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.949887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.950992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.951172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.951251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.951443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.951571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.951799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.951878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80199) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.952892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.953166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80200) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 15 19:10:42.991658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5618fd3a29c0 pid: 80199 (default app) Jul 15 19:10:42.991904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5618fd3a29c0 pid: 80200 (default app) Jul 15 19:10:42.995075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06dd29-f693-4d34-8ed7-a821ee746c6f None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:42.995370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06dd29-f693-4d34-8ed7-a821ee746c6f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:42.995650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06dd29-f693-4d34-8ed7-a821ee746c6f None None] PATH_INFO: `/v3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:43.000118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Fri Jul 15 19:10:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:10:50.649546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:50.650111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:50.650111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:50.654346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:10:50.659537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2497-c936-460b-9166-17beff99730b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:50.659969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2497-c936-460b-9166-17beff99730b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:50.660104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b2497-c936-460b-9166-17beff99730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:50.728299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: INFO dbcounter [None req-735b2497-c936-460b-9166-17beff99730b None None] Registered counter for database keystone Jul 15 19:10:50.749879 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG oslo_db.sqlalchemy.engines [None req-735b2497-c936-460b-9166-17beff99730b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80199) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 19:10:50.755958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writer thread running {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 19:10:50.934126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG passlib.handlers.bcrypt [None req-735b2497-c936-460b-9166-17beff99730b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80199) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 19:10:50.934450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG passlib.handlers.bcrypt [None req-735b2497-c936-460b-9166-17beff99730b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80199) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 19:10:50.988514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-735b2497-c936-460b-9166-17beff99730b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:50.990111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-735b2497-c936-460b-9166-17beff99730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:51.111039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 19:10:51.294904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:51.363006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: INFO dbcounter [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] Registered counter for database keystone Jul 15 19:10:51.383158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG oslo_db.sqlalchemy.engines [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80200) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 15 19:10:51.389502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writer thread running {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 15 19:10:51.511804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cdbddd-8bee-4d56-a697-2f839f43b2cc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:51.516159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:51.516824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:51.517121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:51.517397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:51.525897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:51.586303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 19:10:51.586575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0fbc62-50c5-4b90-a2da-5f2f2eeaebc7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:51.596725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Jul 15 19:10:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 19:10:51.603851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2497-c936-460b-9166-17beff99730b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:51.618835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b2497-c936-460b-9166-17beff99730b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:51.623273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:51.624747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:51.625040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:51.625318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] PATH_INFO: `/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:51.629411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] RBAC: Authorizing `identity:get_project(project_id=98ca6d3d93034fc49fb3c9265ccf0dcb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:51.680554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING keystone.common.rbac_enforcer.enforcer [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 15 19:10:51.681076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e7fd7-b301-4378-81d0-1e59b016c2d3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:51.685362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Fri Jul 15 19:10:51 2022] GET /identity/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:10:53.651108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5cde-7bb5-4925-92f1-60ff2f1c6013 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:53.651646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5cde-7bb5-4925-92f1-60ff2f1c6013 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:53.651646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482e5cde-7bb5-4925-92f1-60ff2f1c6013 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:53.655755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:53.661028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:53.661471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:53.661593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:53.680627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:53.682138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:53.689906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 15 19:10:53.711861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5cde-7bb5-4925-92f1-60ff2f1c6013 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:53.723736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482e5cde-7bb5-4925-92f1-60ff2f1c6013 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:53.728861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646b19f7-c479-44e8-9187-b4e69940249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:53.729504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b19f7-c479-44e8-9187-b4e69940249e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:53.729797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b19f7-c479-44e8-9187-b4e69940249e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:53.730082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646b19f7-c479-44e8-9187-b4e69940249e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:53.737947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b19f7-c479-44e8-9187-b4e69940249e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:53.739744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646b19f7-c479-44e8-9187-b4e69940249e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:53.749402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Fri Jul 15 19:10:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 19:10:53.754961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:53.764532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfa4c3-e3ed-4824-8ce3-085dfe157500 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:53.768842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c759ac-adaf-4898-8f67-cefcc5259c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:53.769707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c759ac-adaf-4898-8f67-cefcc5259c79 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:53.769992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c759ac-adaf-4898-8f67-cefcc5259c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:53.770267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c759ac-adaf-4898-8f67-cefcc5259c79 admin admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:53.771681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c759ac-adaf-4898-8f67-cefcc5259c79 admin admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:53.774175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c759ac-adaf-4898-8f67-cefcc5259c79 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:53.775750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Fri Jul 15 19:10:53 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:10:55.818403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ccf02-c83a-4542-a4ba-34a9f170884f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:55.819480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ccf02-c83a-4542-a4ba-34a9f170884f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:55.819480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ccf02-c83a-4542-a4ba-34a9f170884f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:55.820941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:55.825984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed795-794c-4ff0-a564-1a532e594809 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:55.826372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed795-794c-4ff0-a564-1a532e594809 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:55.826633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed795-794c-4ff0-a564-1a532e594809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:55.843192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c8aed795-794c-4ff0-a564-1a532e594809 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:55.844394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c8aed795-794c-4ff0-a564-1a532e594809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:55.925130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 6/11] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:10:55.940238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca95aab-ae1e-42b8-9cc6-5f49bfea5f66 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:55.940529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca95aab-ae1e-42b8-9cc6-5f49bfea5f66 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:55.940807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca95aab-ae1e-42b8-9cc6-5f49bfea5f66 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:55.942153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 6/12] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:55.947917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:55.948251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:55.948537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:55.949087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca95aab-ae1e-42b8-9cc6-5f49bfea5f66 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:55.965182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:55.966391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:55.982961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.036062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:10:56.058396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.095805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca95aab-ae1e-42b8-9cc6-5f49bfea5f66 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.129268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.130066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.130352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.130627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.141865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:10:56.153107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:10:56.154149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:10:56.155030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:10:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:10:56.157903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640e86a-84ac-438e-b957-a4b636bb53d4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.160196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.193713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.194494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.194839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.195117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.195700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.216885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.265522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.266605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba765ab-1df9-4493-9b4b-881cd0f174d7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.267019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.301336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.301935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.302222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.302502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.303977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:10:56.304310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.305695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.317141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:10:56.322754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.364293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:10:56.370552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.415039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.421528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a34c68-56d7-468d-ab4a-6dff14cd88eb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.422053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.455162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.456017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.456329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.456606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.458208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.460225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.461811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:10:56.467086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.470037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce5dfc4-c98b-4fe4-aaa8-0acd0fd64f5a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.504626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.505414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.505693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.505965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.516752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.518309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:10:56.541006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:10:56.541812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:10:56.542777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:10:56.542777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:10:56.542777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:10:56.542777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:10:56.542777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:10:56.547867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.569845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ae0d59-aaec-4424-94b5-46ae402cf18a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.604117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.604912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.605194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.605469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.606733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.608207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.609813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:10:56.615386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:56.646336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.652550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70affb09-cc8f-42d4-ba37-21a38e199a38 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.655542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:56.687047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.687711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.687991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.688295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.689021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:10:56.689327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.690720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4d1a2-1aba-4949-a7fd-00a3c2acd5c4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.708291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:10:56.717422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b37028-ec07-479f-b418-46db3a2cdae2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:56.758967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:56.759650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:56.759942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:56.760254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:56.761112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.domain.id=default, user.id=477b9e36f0344a75a2c77704966bc3a5) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:10:56.761424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:56.762938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2984e-0af5-437a-9f31-79e5db8190c0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:56.773152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Jul 15 19:10:56 2022] GET /identity/v3/role_assignments?user.id=477b9e36f0344a75a2c77704966bc3a5&scope.domain.id=default&role.id=56597e05853f4660b09a96c651dd0c7a => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:10:59.162964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f949e6-b5e8-4e2f-b685-04001b96da38 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.164330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f949e6-b5e8-4e2f-b685-04001b96da38 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.164820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f949e6-b5e8-4e2f-b685-04001b96da38 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.166386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:59.171994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.172446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.172578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.191882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG passlib.handlers.bcrypt [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80200) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 15 19:10:59.192607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG passlib.handlers.bcrypt [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80200) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 15 19:10:59.223448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af883-4d5c-4a14-86f8-2129232ccee3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.224062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af883-4d5c-4a14-86f8-2129232ccee3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.224564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666af883-4d5c-4a14-86f8-2129232ccee3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.226964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:59.240308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5978a-bc9f-447f-b08b-7d4cec5aebbb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.241586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5978a-bc9f-447f-b08b-7d4cec5aebbb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.242153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5978a-bc9f-447f-b08b-7d4cec5aebbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.249635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:59.251347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:59.277103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-30c5978a-bc9f-447f-b08b-7d4cec5aebbb None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:59.278649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-30c5978a-bc9f-447f-b08b-7d4cec5aebbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:59.362540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 13/24] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:10:59.364690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae603234-7152-477b-8839-8cf66b142015 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.365193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae603234-7152-477b-8839-8cf66b142015 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.366248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae603234-7152-477b-8839-8cf66b142015 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.368600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 14/25] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:59.378064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.379053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.379564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.396212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 12/26] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:10:59.399599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:59.417188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:59.418406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:10:59.451298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:59.541505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 15/27] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:10:59.542772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:59.576293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:59.587654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762d028-a885-4116-961e-63ba0fedde29 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:59.625742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:59.627101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.627563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.628042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.648847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:59.650700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:59.673509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bb00bc-b581-4ae3-98e9-1804eac377f1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ad5eb79c-3179-4806-8093-6a9f2baf9628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:10:59.677744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:10:59.679077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:10:59.679849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:10:59.679849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:10:59.679849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:10:59.679849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:10:59.679849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 13/28] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:10:59.681582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27536da-0b3c-4d34-9a8f-32cff149a61a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.681938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27536da-0b3c-4d34-9a8f-32cff149a61a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.682290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27536da-0b3c-4d34-9a8f-32cff149a61a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.683647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 14/29] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:10:59.684437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27536da-0b3c-4d34-9a8f-32cff149a61a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:59.726549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:59.727800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.728569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.729323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.746668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:10:59.759006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:10:59.759855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:10:59.760621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:10:59.762179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 16/30] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:10:59.763766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:10:59.794608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:59.820848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27536da-0b3c-4d34-9a8f-32cff149a61a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:59.861278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:59.862408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.862908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.863386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.864400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:59.866612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:59.882730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:10:59.903044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5396ae-4d77-4316-960d-8e85f233b659 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:10:59.940938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:10:59.941933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.942397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.942839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.943751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:10:59.944241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:10:59.945777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc2d2e-0821-48d7-8f48-3d0ba3464872 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:10:59.958148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 17/31] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:10:59.959881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:10:59.960361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:10:59.960831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:10:59.979950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:10:59.981396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:00.027411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 15/32] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Jul 15 19:10:59 2022] POST /identity/v3/roles => generated 219 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:00.029132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.060642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 18/33] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:00.068435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.155454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ebd25-8ec3-4416-b295-7c026d69c8c3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.167712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:00.197871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.198556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.198928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.199284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.200160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:00.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.202324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.208456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:00.214374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 16/34] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:00.217068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.234421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be9d01-77a0-46d8-83ab-97c501d5d282 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.306472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493fdd4-ea02-4809-a21a-476a03325267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.308573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493fdd4-ea02-4809-a21a-476a03325267 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.309107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493fdd4-ea02-4809-a21a-476a03325267 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.309591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493fdd4-ea02-4809-a21a-476a03325267 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.310547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4493fdd4-ea02-4809-a21a-476a03325267 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.312466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4493fdd4-ea02-4809-a21a-476a03325267 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.347487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369f60d-c24d-46b6-b047-0ae8a076c5bc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.388925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.389674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.390054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.390342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.391220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.392819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.408648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 17/35] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Jul 15 19:11:00 2022] POST /identity/v3/roles => generated 225 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:00.409215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.479278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:00.513763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 19/36] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:11:00 2022] POST /identity/v3/projects => generated 301 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:00.516381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb330a-c248-44ca-944c-5516783839f1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.516949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb330a-c248-44ca-944c-5516783839f1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.517648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb330a-c248-44ca-944c-5516783839f1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.519598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 20/37] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:00.526299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.526299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.526299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.557230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78292712-5ff1-4105-a627-0c4dbd08ef90 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.557448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:00.558963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:00.608823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.610923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.611539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.612026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.614382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.616617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.618999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 18/38] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:00 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:00.625489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.706067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 21/39] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:00.729567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.750989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84e64e-9439-4780-bfb3-2ff2a3141466 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.782649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:00.791504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.792354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.792642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.792926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.794218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.795739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.797419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 19/40] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:00.803902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:00.809005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:00.864738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d18744-4361-4557-a529-8f9e6462ba18 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.900777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9790a4-090c-4c4e-804c-98061c7c9aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.901453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9790a4-090c-4c4e-804c-98061c7c9aad None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.901745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9790a4-090c-4c4e-804c-98061c7c9aad None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.902026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9790a4-090c-4c4e-804c-98061c7c9aad None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.902776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9790a4-090c-4c4e-804c-98061c7c9aad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.904340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9790a4-090c-4c4e-804c-98061c7c9aad None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.914826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec04099d-9cc6-4f61-b99e-f59c84b35554 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:00.918292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 22/41] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Jul 15 19:11:00 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:00.919791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580c6ef-c56b-40ec-bfec-e9be9517a15e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.920103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580c6ef-c56b-40ec-bfec-e9be9517a15e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.920398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e580c6ef-c56b-40ec-bfec-e9be9517a15e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.924131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 23/42] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:00.930038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.930327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.930919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.959231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:00.960746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:00.961035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:00.961317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] PATH_INFO: `/v3/domains/default/users/477b9e36f0344a75a2c77704966bc3a5/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:00.963561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=477b9e36f0344a75a2c77704966bc3a5, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:00.965128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:00.992210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 20/43] 10.222.0.17 () {58 vars in 1361 bytes} [Fri Jul 15 19:11:00 2022] PUT /identity/v3/domains/default/users/477b9e36f0344a75a2c77704966bc3a5/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:11:01.065893 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:01.067170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:01.146904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 24/44] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:01.169655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:01.227353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:01.273888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf2b10-3f18-436e-8d70-fcd5bee281af None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:01.307149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd710c8e-a495-446c-9e43-05320e1412ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:01.307951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd710c8e-a495-446c-9e43-05320e1412ac None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:01.308269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd710c8e-a495-446c-9e43-05320e1412ac None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:01.308552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd710c8e-a495-446c-9e43-05320e1412ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:01.309729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd710c8e-a495-446c-9e43-05320e1412ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:01.311176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd710c8e-a495-446c-9e43-05320e1412ac None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:01.312840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 21/45] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:01.318352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:01.364703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:01.418337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f05d7b-b04b-4d9f-8129-72dd3061e187 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:01.451730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23d7559-99bd-4e98-8bce-721104ab9cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:01.452433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7559-99bd-4e98-8bce-721104ab9cc9 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:01.452725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7559-99bd-4e98-8bce-721104ab9cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:01.453006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23d7559-99bd-4e98-8bce-721104ab9cc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:01.453735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d7559-99bd-4e98-8bce-721104ab9cc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:01.455215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23d7559-99bd-4e98-8bce-721104ab9cc9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:01.480001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 25/46] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:11:01 2022] POST /identity/v3/projects => generated 312 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:03.026569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d58cf-0400-4c8d-a345-3ffdb504b52d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.027063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d58cf-0400-4c8d-a345-3ffdb504b52d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.027063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d58cf-0400-4c8d-a345-3ffdb504b52d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.028453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 22/47] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:03.033584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.033960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.034262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.051320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:03.052526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:03.118158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 26/48] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:03.140273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d58cf-0400-4c8d-a345-3ffdb504b52d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.241197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d58cf-0400-4c8d-a345-3ffdb504b52d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.264734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:03.276136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.276982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.277221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.277502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.287460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:03.295599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:03.296451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:03.297266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 23/49] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:03.302640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.331137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:03.405110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f76979-1c2a-4f1c-95c4-3b586222cd81 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.439613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.440323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.440615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.440898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.441655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:03.441966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:03.443376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e4c98-09df-429b-9fa8-87b5b3659d83 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:03.455438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 27/50] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:03.462128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.468628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03661a63-a443-4a7e-a254-5568d93e37a1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.468922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03661a63-a443-4a7e-a254-5568d93e37a1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.469206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03661a63-a443-4a7e-a254-5568d93e37a1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.470542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 28/51] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:03.476302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.476591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.476870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.492589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:03.493784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:03.559328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 29/52] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:03.564399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba851983-94bc-4a0e-9b17-cd0c16a94957 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.582714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.600855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.601739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.602025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.602304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.603523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:03.604248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:03.605092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:03.606730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 24/53] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:03 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:03.612234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.668966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:03.682920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de161b8-3202-46a3-afdb-d9fc387d633a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.716014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb7b02e-15cd-4dfc-8f32-a1468a4c7264 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.717366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.718272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.718559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.718850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.720048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:03.721531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:03.723138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 30/54] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:03.728867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.750841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.751640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.751926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.752243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.753425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:03.754929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:03.757101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 25/55] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:03.763073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:03.820728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:03.832516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666c39a1-bf4a-446c-bf8a-48b506573131 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.864438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47254c6-f6ff-4bb7-a406-9260a5ac68a3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:03.866974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8536053e-5076-42f3-bad5-2a9ee227d00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.867638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536053e-5076-42f3-bad5-2a9ee227d00e None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.867923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536053e-5076-42f3-bad5-2a9ee227d00e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.868236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8536053e-5076-42f3-bad5-2a9ee227d00e None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.868939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536053e-5076-42f3-bad5-2a9ee227d00e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:03.870384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8536053e-5076-42f3-bad5-2a9ee227d00e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:03.885147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:03.897541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 31/56] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Jul 15 19:11:03 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:11:03.900524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:03.901227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:03.901519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:03.901804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:03.902694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.domain.id=default, user.id=477b9e36f0344a75a2c77704966bc3a5) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:03.903014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:03.904615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a28bce2-6ec9-4e5a-8796-e4fa0976a2e3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:03.911458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 26/57] 10.222.0.17 () {56 vars in 1322 bytes} [Fri Jul 15 19:11:03 2022] GET /identity/v3/role_assignments?user.id=477b9e36f0344a75a2c77704966bc3a5&scope.domain.id=default&role.id=56597e05853f4660b09a96c651dd0c7a => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:05.952745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ae669-f845-4f83-b1ab-32f96532d85d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:05.953267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ae669-f845-4f83-b1ab-32f96532d85d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:05.953356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ae669-f845-4f83-b1ab-32f96532d85d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:05.954836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 32/58] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:05.959860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:05.960254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:05.960449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:05.978164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:05.979672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:06.150074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 27/59] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:06.174244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ae669-f845-4f83-b1ab-32f96532d85d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:06.272934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ae669-f845-4f83-b1ab-32f96532d85d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:06.305013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81599bd8-be0e-4595-bd5e-60e3bff96353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:06.305819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81599bd8-be0e-4595-bd5e-60e3bff96353 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:06.306085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81599bd8-be0e-4595-bd5e-60e3bff96353 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:06.306367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81599bd8-be0e-4595-bd5e-60e3bff96353 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:06.307539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81599bd8-be0e-4595-bd5e-60e3bff96353 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:06.309015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81599bd8-be0e-4595-bd5e-60e3bff96353 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:06.310613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 33/60] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:06.316122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:06.413068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7448a63-164b-482d-bcf2-74352d019889 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:06.426082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:06.446927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77e9839-4ae0-4ad5-a11e-90d8f7543b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:06.447595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e9839-4ae0-4ad5-a11e-90d8f7543b49 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:06.447885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e9839-4ae0-4ad5-a11e-90d8f7543b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:06.448195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77e9839-4ae0-4ad5-a11e-90d8f7543b49 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:06.448905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e9839-4ae0-4ad5-a11e-90d8f7543b49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:06.450372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77e9839-4ae0-4ad5-a11e-90d8f7543b49 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:06.474370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 28/61] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:06 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:09.520609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d4c5-0350-4bb7-9bad-6758fc376ee1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.521115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d4c5-0350-4bb7-9bad-6758fc376ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.521222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9d4c5-0350-4bb7-9bad-6758fc376ee1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:09.523826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 34/62] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:09.542312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5086-0651-4978-8f73-84cdb194bbea None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.543232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5086-0651-4978-8f73-84cdb194bbea None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.543724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5086-0651-4978-8f73-84cdb194bbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:09.570182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-852f5086-0651-4978-8f73-84cdb194bbea None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:09.571750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-852f5086-0651-4978-8f73-84cdb194bbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:09.655720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 29/63] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:09.694956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d4c5-0350-4bb7-9bad-6758fc376ee1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:09.716050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:09.844244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7327c3-727f-4926-9634-71e970cf3643 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.844874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7327c3-727f-4926-9634-71e970cf3643 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.845354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7327c3-727f-4926-9634-71e970cf3643 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:09.846234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9d4c5-0350-4bb7-9bad-6758fc376ee1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:09.847149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 30/64] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:09.857489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aacbde-2671-4de6-91e7-64661d5c1a06 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.858053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aacbde-2671-4de6-91e7-64661d5c1a06 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.858550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aacbde-2671-4de6-91e7-64661d5c1a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:09.887050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef6035-2c5d-49bb-9331-f6d11e7b115a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:09.888146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef6035-2c5d-49bb-9331-f6d11e7b115a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.888439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef6035-2c5d-49bb-9331-f6d11e7b115a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.888718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef6035-2c5d-49bb-9331-f6d11e7b115a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:09.901048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-19aacbde-2671-4de6-91e7-64661d5c1a06 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:09.902240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-19aacbde-2671-4de6-91e7-64661d5c1a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:09.909372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef6035-2c5d-49bb-9331-f6d11e7b115a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ccef6035-2c5d-49bb-9331-f6d11e7b115a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:09.917562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:09.918389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:09.919130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 35/65] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:09.920109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf11eb-b1bc-4c36-b1e4-8cc09a030704 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.920540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf11eb-b1bc-4c36-b1e4-8cc09a030704 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.920992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf11eb-b1bc-4c36-b1e4-8cc09a030704 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:09.922394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 36/66] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:09.927224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf11eb-b1bc-4c36-b1e4-8cc09a030704 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:09.987447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 31/67] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:09.988993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:09.989425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:09.989844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.000317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.007690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:10.009108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:10.035400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.039420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf11eb-b1bc-4c36-b1e4-8cc09a030704 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.076569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.077513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.077945 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.078362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.079268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:10.079727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:10.081397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:10.081875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 32/68] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:10.082874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.103146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 37/69] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:10.108704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.185292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598179b-3a2e-49bd-bc1e-e7388e9cf2de None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.214202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3a182-9f27-4ca6-bcd0-93ab0b323e94 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.218554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c90c00-aa9e-47bd-8d96-05040cc9508b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.219343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90c00-aa9e-47bd-8d96-05040cc9508b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.219627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90c00-aa9e-47bd-8d96-05040cc9508b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.219904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c90c00-aa9e-47bd-8d96-05040cc9508b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.230178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c90c00-aa9e-47bd-8d96-05040cc9508b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-41c90c00-aa9e-47bd-8d96-05040cc9508b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:10.237840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:10.238800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:10.239544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 33/70] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:10.241180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78da62-0e1f-466b-b821-0f9486edeaf1 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.241180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78da62-0e1f-466b-b821-0f9486edeaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.241425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78da62-0e1f-466b-b821-0f9486edeaf1 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.242570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 34/71] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:10.244208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0946-9021-409c-8fbd-cd82d1dfe87d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.244208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0946-9021-409c-8fbd-cd82d1dfe87d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.244208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed0946-9021-409c-8fbd-cd82d1dfe87d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.246960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 35/72] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:10.246960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0946-9021-409c-8fbd-cd82d1dfe87d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.249268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.256931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db08e9-2c5c-4847-81ef-9880662e433c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.257922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db08e9-2c5c-4847-81ef-9880662e433c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.258210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db08e9-2c5c-4847-81ef-9880662e433c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.258489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db08e9-2c5c-4847-81ef-9880662e433c None admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.259715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db08e9-2c5c-4847-81ef-9880662e433c None admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:10.261240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db08e9-2c5c-4847-81ef-9880662e433c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:10.262886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:10.264107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.264410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.264694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.274990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.282687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:10.283881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:10.368174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 39/74] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:10.368344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed0946-9021-409c-8fbd-cd82d1dfe87d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.368430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.391079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7551d0-ce06-450b-a2bf-6cf1ac19bdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.391871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7551d0-ce06-450b-a2bf-6cf1ac19bdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.392185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7551d0-ce06-450b-a2bf-6cf1ac19bdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.392466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7551d0-ce06-450b-a2bf-6cf1ac19bdd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.393682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7551d0-ce06-450b-a2bf-6cf1ac19bdd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:10.395186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7551d0-ce06-450b-a2bf-6cf1ac19bdd2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:10.396865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 36/75] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:10.398185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.398474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.398755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.415456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:10.416688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:10.460637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b9f71-e5bb-48d4-b82f-0da9ae2f553b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.482247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 37/76] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:10.483076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.493273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.501729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.502405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.502691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.502979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.503615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.503740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:10.504030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:10.505464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:10.514103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 40/77] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:10.514898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.585142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6768563-2173-4c78-920f-e5a0cdf49e05 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.615748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d620362-94d7-45fa-bd9e-ae9fde91e3ba None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.618082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.618792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.619026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.619308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.649688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.650508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.650804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.651067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.662730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:10.671232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:10.672325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:10.673071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:10.674212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 41/78] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:10.679241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.699255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:10.700762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:10.718600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.724388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.778225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad081df7-2c79-43e7-b5c6-2d35e7d586e7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.805347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 38/79] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:10.806430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.815450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.816156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.816446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.816727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.817433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:10.819057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:10.845197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 42/80] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:10 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:10.846000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:10.920019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c63469-a731-4f55-879e-155775c8d473 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.939224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.959454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:10.960435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:10.960829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:10.961108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:10.963808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5da6f-fa27-4c4a-8736-9b977f0d4e85 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:10.971735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:10.974476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:10.982711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:10.983788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 39/81] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:10.988858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.012496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.013645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.014057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.014497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.035636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.037768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.039707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 43/82] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.040414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.110667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a417d2-6ac1-410d-a315-aed472eccdbd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.155462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.156342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.156446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.156731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.157987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:11.157987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.159194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.169003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 40/83] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.170095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d8097-3fe8-4909-a1cc-a0c68c9629c5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.170464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.206495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.207370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.207651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.207931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.209357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.210971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.212946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 44/84] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.213536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.224138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:11.245551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:11.300657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d2a354-ad52-44fe-a4e9-2f79e8195085 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.320372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71db703-046e-4edf-9020-90cec15f6067 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.348350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.349882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.350352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.350807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.352201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:11.352998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.354562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.356902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.357636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.357991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.358337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.365688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 41/85] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.366818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.452611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.454745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.477692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf10aaa-a5dc-4236-a30b-fe2fc0be7aa8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.480457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:11.495250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:11.519308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.520636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.521146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.521677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.523120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.524818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.526796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 42/86] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.527774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.594290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 45/87] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:11.595579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.635479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f102382-6d93-4359-95aa-8681669b566d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.671379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.672425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.673044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.673512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.674545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=477b9e36f0344a75a2c77704966bc3a5) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:11.675069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.676756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.685109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 43/88] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/role_assignments?user.id=477b9e36f0344a75a2c77704966bc3a5&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.686223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.711877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a4445-2d73-4362-a4f1-931d465895cb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.726334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:11.752524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:11.771244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.775032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.775032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.775032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.781127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.782662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.784479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 46/89] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:11.786410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:11.815258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a770f3-048c-4c19-a616-1183d8e2eab9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.855480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:11.856620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:11.857125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:11.857644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:11.934075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:11.936686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:11.966390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a714d-cead-4445-ba98-210be7f981d8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:11.971800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:12.011607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:12.012531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:12.012821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:12.013107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:12.014336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:12.015822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:12.024597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 47/90] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:12.025418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:12.054303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 44/91] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:12.055523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:12.159840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2ad1-db15-47fe-8141-cbe35a901041 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:12.164456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:12.195798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb823663-49d6-4a72-87da-6dabd886c5eb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:12.197808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:12.198691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:12.199171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:12.199639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:12.200733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=30cadd9bcfd84eaba7b31000da7993e1) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:12.201241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:12.202953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:12.209788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 45/92] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:12 2022] GET /identity/v3/role_assignments?user.id=30cadd9bcfd84eaba7b31000da7993e1&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:12.211123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:12.211819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:12.260441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:12.261044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:12.261339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:12.261628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:12.377651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0645e8e-7a9e-4dcd-8cb5-bba3d0ee3276 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:12.444985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:12.446295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:12.446778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:12.447275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:12.448908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:12.450624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:12.452691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:12 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:12.456107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:12.456107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:12.458693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:12.549592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:12.588724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8daf49-45bd-488d-b9b0-53237d889fa9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:12.625641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:12.626640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:12.627146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:12.627626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:12.628426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:12.629009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] RBAC: Adding query filter params (role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=30cadd9bcfd84eaba7b31000da7993e1) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:12.629538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:12.631320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:12.637579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 47/94] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:12 2022] GET /identity/v3/role_assignments?user.id=30cadd9bcfd84eaba7b31000da7993e1&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:12.689353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 48/95] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:12.695239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:12.822825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fea6aa-1266-44bf-9670-4d61c3103f08 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:12.857675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:12.867071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190b72d-8bf8-418d-846d-05133fd08415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:12.867970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b72d-8bf8-418d-846d-05133fd08415 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:12.868296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b72d-8bf8-418d-846d-05133fd08415 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:12.868581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190b72d-8bf8-418d-846d-05133fd08415 None admin] PATH_INFO: `/v3/projects/895bb12f67af4e8490a762a628b8475e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:12.869860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b72d-8bf8-418d-846d-05133fd08415 None admin] RBAC: Authorizing `identity:get_project(project_id=895bb12f67af4e8490a762a628b8475e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:12.871348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190b72d-8bf8-418d-846d-05133fd08415 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:12.873172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 48/96] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:12 2022] GET /identity/v3/projects/895bb12f67af4e8490a762a628b8475e => generated 312 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:12.879361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:13.005989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dda92f-20de-448a-ab55-ca0baa0a3f40 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:13.045998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:13.046667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:13.046957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:13.047240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:13.048106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=895bb12f67af4e8490a762a628b8475e, user.id=30cadd9bcfd84eaba7b31000da7993e1) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:13.048426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:13.049890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed2505-7817-4443-8098-82dac3215774 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:13.060254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 49/97] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:12 2022] GET /identity/v3/role_assignments?user.id=30cadd9bcfd84eaba7b31000da7993e1&scope.project.id=895bb12f67af4e8490a762a628b8475e&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:13.321655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9943d-752b-4b58-ac2d-b0d860ea26b2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:13.321951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9943d-752b-4b58-ac2d-b0d860ea26b2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:13.322243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a9943d-752b-4b58-ac2d-b0d860ea26b2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:13.323638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 49/98] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:13.329127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:13.329424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:13.329714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:13.356154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:13.357469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:13.396832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:13.446865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 50/99] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:13.469519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9943d-752b-4b58-ac2d-b0d860ea26b2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:13.593452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a9943d-752b-4b58-ac2d-b0d860ea26b2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:13.647370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:13.648293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:13.648469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:13.648748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:13.659212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:13.669439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:13.670272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:13.671417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 50/100] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:13.676440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:13.856204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c89429-5efd-41a8-aa9e-8376503e82dd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:13.932196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:13.932943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:13.933209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:13.933457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:13.934214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:13.934518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:13.935929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0dbd7f-a3a9-4e6f-8bc4-657dd6fc251a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:13.945671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 51/101] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:13.951739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:14.024847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8a39-3711-49cb-af51-999a840b53ce None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:14.025160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8a39-3711-49cb-af51-999a840b53ce None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:14.025441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f8a39-3711-49cb-af51-999a840b53ce None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:14.026829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 52/102] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:14.039502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:14.039798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:14.040169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:14.062314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:14.068957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:14.107558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:14.133704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:14.204964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326fee-cd37-48f7-ae63-ec77572f320e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:14.281391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:14.282347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:14.282632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:14.282910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] PATH_INFO: `/v3/users/9c192523a59d410aafadaea708fabd52` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:14.292297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 53/103] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:14.308929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] RBAC: Authorizing `identity:get_user(user_id=9c192523a59d410aafadaea708fabd52)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:14.310413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:14.320359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 51/104] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:13 2022] GET /identity/v3/users/9c192523a59d410aafadaea708fabd52 => generated 289 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:14.321763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:14.373251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:14.583775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe0594-d679-4d0a-a02d-ad67893a9a30 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:14.636193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:14.665605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:14.666436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:14.666729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:14.667002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:14.668329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:14.669822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:14.671537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 52/105] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:14.681329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:14.683112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ad884-cc8f-4257-8d3f-481d80be17f4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:14.748115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:14.748115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:14.748115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:14.748115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:14.789808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:14.856402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:14.857899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:14.884487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc293b-3cae-4670-9021-e1dff6672aa5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:14.952890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2aaee-2a54-48c1-9af0-2327e1fa010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:14.953879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2aaee-2a54-48c1-9af0-2327e1fa010c None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:14.954363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2aaee-2a54-48c1-9af0-2327e1fa010c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:14.954882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2aaee-2a54-48c1-9af0-2327e1fa010c None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:14.955798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2aaee-2a54-48c1-9af0-2327e1fa010c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:14.957466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2aaee-2a54-48c1-9af0-2327e1fa010c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:15.009271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 53/106] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:11:14 2022] POST /identity/v3/projects => generated 302 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:11:15.011093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267b457-041b-468c-9206-89eeda54557e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.011607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267b457-041b-468c-9206-89eeda54557e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.012117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267b457-041b-468c-9206-89eeda54557e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.013701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 54/107] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:15.023237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.023870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.024445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.046788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 54/108] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 725 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:15.052638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.058030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:15.063780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:15.124785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:15.156448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:15.198578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 55/109] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 15 19:11:15.221311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.234365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b98b8e-e5e0-4a2f-9e31-52231b5b76eb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:15.302110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807779d-dd42-48f9-bdd5-710df7235b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:15.303530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807779d-dd42-48f9-bdd5-710df7235b25 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.304099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807779d-dd42-48f9-bdd5-710df7235b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.304404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807779d-dd42-48f9-bdd5-710df7235b25 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.305835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807779d-dd42-48f9-bdd5-710df7235b25 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:15.308128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2807779d-dd42-48f9-bdd5-710df7235b25 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:15.310950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 55/110] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:15.312630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191db7-3f5a-465b-9442-63306ef5f129 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.312923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191db7-3f5a-465b-9442-63306ef5f129 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.313207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd191db7-3f5a-465b-9442-63306ef5f129 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.314590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 56/111] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:15.320218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd191db7-3f5a-465b-9442-63306ef5f129 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.367348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb9fc7c-35c7-483e-bdcf-de3536bd435b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:15.407947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdb8d3c-2db0-4198-b320-7282727b6ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:15.408930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb8d3c-2db0-4198-b320-7282727b6ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.409369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb8d3c-2db0-4198-b320-7282727b6ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.409741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdb8d3c-2db0-4198-b320-7282727b6ea3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.421686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdb8d3c-2db0-4198-b320-7282727b6ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4bdb8d3c-2db0-4198-b320-7282727b6ea3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:15.430039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:15.430904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:15.431657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 56/112] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:15.435040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.435040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.435040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.445765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd191db7-3f5a-465b-9442-63306ef5f129 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:15.452123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:15.453360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:15.476043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:15.481136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:15.485754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:15.486543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.486831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.487208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.488191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] RBAC: Adding query filter params (role.id=7bc8b631d47745919f94cb9a1362c22e, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=9c192523a59d410aafadaea708fabd52) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:15.488565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:15.490890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:15.499251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 57/113] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/role_assignments?user.id=9c192523a59d410aafadaea708fabd52&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=7bc8b631d47745919f94cb9a1362c22e => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:15.505039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.532678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 57/114] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:15.556893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.671103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dccd3ad-0810-4a42-9f87-0a84dab53b57 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:15.714257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:15.715074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.715356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.715636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.727972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:15.736503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:15.737378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:15.738128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 58/115] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:15.743691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.774717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:15.776825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a666b40-1288-4a78-a21b-8d477949fa71 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:15.855727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:15.856460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.856749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.857033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.857406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd33eb-4f3e-45ee-ad04-16e4cf3294a5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:15.857803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:15.858113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:15.859535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:15.875568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 58/116] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:15.881472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.899299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:15.900358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.900465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.900661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.901436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:15.901745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:15.903181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5ee5b-b8c9-4b1f-92a7-9afb8b1fe051 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:15.913498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 59/117] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:15.914647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74a30c-d425-4303-9a4d-3d00337eb596 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:15.914936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74a30c-d425-4303-9a4d-3d00337eb596 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:15.915213 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74a30c-d425-4303-9a4d-3d00337eb596 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:15.921541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 60/118] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:15.922333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74a30c-d425-4303-9a4d-3d00337eb596 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:15.963711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.020668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.052751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74a30c-d425-4303-9a4d-3d00337eb596 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.077511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb7733-c3eb-490c-8721-13834ce70c7f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.118579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118475e4-eeb4-46bd-a594-a19ddfe0e4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.119818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118475e4-eeb4-46bd-a594-a19ddfe0e4ad None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.120333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118475e4-eeb4-46bd-a594-a19ddfe0e4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.120848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118475e4-eeb4-46bd-a594-a19ddfe0e4ad None admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.122330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118475e4-eeb4-46bd-a594-a19ddfe0e4ad None admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.124287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118475e4-eeb4-46bd-a594-a19ddfe0e4ad None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.126160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 59/119] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:16.127675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.128194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.128694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.134816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.135679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.135961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.136268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.138869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.140353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.141995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 61/120] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:15 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:16.142751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.156578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:16.157763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:16.245487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 60/121] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:16.246710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.267603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dfe8f-2388-49b2-91e9-ea287b8d38a0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.303997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a14a07-4405-4a0f-96e9-4e052496db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.304829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a14a07-4405-4a0f-96e9-4e052496db8f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.305118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a14a07-4405-4a0f-96e9-4e052496db8f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.305400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a14a07-4405-4a0f-96e9-4e052496db8f None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.306607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a14a07-4405-4a0f-96e9-4e052496db8f None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.308064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a14a07-4405-4a0f-96e9-4e052496db8f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.308400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.309811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 62/122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:16 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:16.315704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3807a-b403-46ec-94be-d4b79cecb80d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.315998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3807a-b403-46ec-94be-d4b79cecb80d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.316558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3807a-b403-46ec-94be-d4b79cecb80d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.322901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 63/123] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:16.323614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3807a-b403-46ec-94be-d4b79cecb80d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.341291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.375696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da53640-0e88-43a2-9c96-e24205f9cd0c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.413073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.414056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.414411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.414761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.416049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.417629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.419775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 61/124] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:16 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:16.420775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.463019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3807a-b403-46ec-94be-d4b79cecb80d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.510705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a05f077-edd2-4d79-b05e-342340b0c30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.511666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05f077-edd2-4d79-b05e-342340b0c30b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.512026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05f077-edd2-4d79-b05e-342340b0c30b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.512423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a05f077-edd2-4d79-b05e-342340b0c30b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.523928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a05f077-edd2-4d79-b05e-342340b0c30b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3a05f077-edd2-4d79-b05e-342340b0c30b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:16.532122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:16.533861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:16.535475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:16.536736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70139419-eafc-4c55-a04a-088b3b149f24 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.537409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 64/125] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:16.537540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.544149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.544149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.550410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.557627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:16.559005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:16.579897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.581100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.582097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.582561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/477b9e36f0344a75a2c77704966bc3a5/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.586094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9, user_id=477b9e36f0344a75a2c77704966bc3a5, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.588327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.607991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 62/126] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:16 2022] PUT /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/477b9e36f0344a75a2c77704966bc3a5/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:16.609569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.674434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 65/127] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:16.675443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.686107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.740531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936df8a3-1c6b-40b6-9565-3a95848d18b5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.782129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.783448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.783950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.784615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/30cadd9bcfd84eaba7b31000da7993e1/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.787920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9, user_id=30cadd9bcfd84eaba7b31000da7993e1, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.791094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.809522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 63/128] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:16 2022] PUT /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/30cadd9bcfd84eaba7b31000da7993e1/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:16.811461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:16.861607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:16.909395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441bd82-fd2e-4317-8b23-eebb2a105e45 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.954307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:16.955165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:16.955454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:16.955733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:16.956721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:16.956921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:16.958349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:16.963990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f58b1f-1f44-4ad5-9bfb-c62b424ba374 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:16.969068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 66/129] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:16.977194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.008920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.010680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.011185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.011734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.026134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:17.035655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:17.036621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:17.037982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:17.040546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 64/130] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:17.047357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.125200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:17.161463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:17.172131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a480a4-936f-48e7-9d9a-e4c7b3ebfbd5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:17.177553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357754-fdeb-4eb5-9a76-1764665f8649 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:17.250110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.250792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.251076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.251353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.252120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:17.252435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:17.253840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:17.255118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.256592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.256875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.257187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.258463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:17.259987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:17.263071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 67/131] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:16 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:17.270972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.278697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 65/132] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:17.284561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.418647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb0aba7-f9ee-48fd-af6d-4a22255721de None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:17.454881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.455683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.455968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.456271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.457479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:17.458960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:17.460880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 68/133] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:17 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:17.466289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.469027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:17.470420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a476c379-58cc-4544-b9c8-707f0e70833f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:17.531413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.532345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.532632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.532910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.537089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:17.538758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:17.544107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 66/134] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:17 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:17.546653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.597652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc9300-02f0-4aa8-95b5-10bb78f57a94 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:17.605093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:17.657750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.658970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.659306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.659591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/30cadd9bcfd84eaba7b31000da7993e1/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.666700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9, user_id=30cadd9bcfd84eaba7b31000da7993e1, role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:17.668200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:17.696543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 69/135] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:17 2022] PUT /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/30cadd9bcfd84eaba7b31000da7993e1/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:11:17.763093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccccc3b-21f6-41e1-903f-11071bed8876 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:17.805197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ad83c-4baa-4401-acb7-8565a0e523bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:17.806077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad83c-4baa-4401-acb7-8565a0e523bf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:17.806359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad83c-4baa-4401-acb7-8565a0e523bf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:17.806638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ad83c-4baa-4401-acb7-8565a0e523bf None admin] PATH_INFO: `/v3/projects/895bb12f67af4e8490a762a628b8475e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:17.808673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ad83c-4baa-4401-acb7-8565a0e523bf None admin] RBAC: Authorizing `identity:get_project(project_id=895bb12f67af4e8490a762a628b8475e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:17.810236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ad83c-4baa-4401-acb7-8565a0e523bf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:17.812065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 67/136] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:17 2022] GET /identity/v3/projects/895bb12f67af4e8490a762a628b8475e => generated 312 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:17.817916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:17.925453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:17.978608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd9bd6-75e9-412a-9a01-ae95d2fb13a4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:18.024677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9e8e1-6d2f-4c83-bad0-d0a880a64313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:18.025588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e8e1-6d2f-4c83-bad0-d0a880a64313 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.025868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e8e1-6d2f-4c83-bad0-d0a880a64313 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.026132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9e8e1-6d2f-4c83-bad0-d0a880a64313 None admin] PATH_INFO: `/v3/projects/895bb12f67af4e8490a762a628b8475e/users/30cadd9bcfd84eaba7b31000da7993e1/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.033549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9e8e1-6d2f-4c83-bad0-d0a880a64313 None admin] RBAC: Authorizing `identity:create_grant(project_id=895bb12f67af4e8490a762a628b8475e, user_id=30cadd9bcfd84eaba7b31000da7993e1, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:18.035036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9e8e1-6d2f-4c83-bad0-d0a880a64313 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:18.069942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 70/137] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:17 2022] PUT /identity/v3/projects/895bb12f67af4e8490a762a628b8475e/users/30cadd9bcfd84eaba7b31000da7993e1/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:18.174768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fb06d-0d21-44b6-80a7-ce4de2fadfcd None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.175299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fb06d-0d21-44b6-80a7-ce4de2fadfcd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.175381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296fb06d-0d21-44b6-80a7-ce4de2fadfcd None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.176853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 68/138] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:18.184114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.184427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.184704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.209819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:18.211180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:18.267522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf889-d618-4ef0-85ce-da0a644df67c None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.267814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf889-d618-4ef0-85ce-da0a644df67c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.268114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adcf889-d618-4ef0-85ce-da0a644df67c None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.269537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 69/139] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:18.278398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.278691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.278970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.298439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:18.299851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 71/140] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:18.299958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:18.357672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:18.391124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 70/141] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:18.420694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:18.450138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:18.456643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:18.539216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc499bb4-d25a-431a-ade4-d28f0c99391d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:18.577892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:18.578755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.579040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.579317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.586052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ba144-c1f6-48ea-a238-c576d6c409a6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:18.590591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:18.599081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:18.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:18.601885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 72/142] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:18.607902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:18.662803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:18.664124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.664658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.665141 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.672882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:18.674771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:18.676869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 71/143] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:18.699352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:18.731515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ff51e-e90b-40e3-b011-a41c05ecbe37 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:18.750191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:18.779841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:18.779841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.779841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.779841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.780228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:18.780228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:18.780228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:18.787927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 73/144] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:18.793353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:18.898895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3efc0-8e16-4b2d-b694-916cfe245a42 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:18.906887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:18.925102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d47cbe-af01-4d5b-870f-b6370831b0c4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:18.939233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:18.940148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:18.940436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:18.940704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] PATH_INFO: `/v3/users/9c192523a59d410aafadaea708fabd52` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:18.942284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] RBAC: Authorizing `identity:get_user(user_id=9c192523a59d410aafadaea708fabd52)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:18.943954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:18.945670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 74/145] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:18 2022] GET /identity/v3/users/9c192523a59d410aafadaea708fabd52 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:18.956418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:19.004605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:19.005287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.005725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.006050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.006743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:19.008255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:19.034474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:19.058723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 72/146] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:18 2022] POST /identity/v3/users => generated 283 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:11:19.114058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe13e00-6154-4883-ab7a-034b52c820f5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:19.161753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc19e13-aaa8-4f11-92c3-46220a73adf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:19.162728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc19e13-aaa8-4f11-92c3-46220a73adf7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.163024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc19e13-aaa8-4f11-92c3-46220a73adf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.163296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc19e13-aaa8-4f11-92c3-46220a73adf7 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.164583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc19e13-aaa8-4f11-92c3-46220a73adf7 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:19.166111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc19e13-aaa8-4f11-92c3-46220a73adf7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:19.167889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 75/147] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:18 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:19.173718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:19.292182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e215ae-4c49-4d49-bcf0-3c89c9b7149a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:19.336540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524ad97b-3aa7-4df4-ad73-1499d551a8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:19.337414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ad97b-3aa7-4df4-ad73-1499d551a8c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.337699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ad97b-3aa7-4df4-ad73-1499d551a8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.337978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524ad97b-3aa7-4df4-ad73-1499d551a8c6 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/9c192523a59d410aafadaea708fabd52/roles/7bc8b631d47745919f94cb9a1362c22e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.344624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ad97b-3aa7-4df4-ad73-1499d551a8c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9, user_id=9c192523a59d410aafadaea708fabd52, role_id=7bc8b631d47745919f94cb9a1362c22e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:19.346126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524ad97b-3aa7-4df4-ad73-1499d551a8c6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:19.373116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 73/148] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:19 2022] PUT /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/users/9c192523a59d410aafadaea708fabd52/roles/7bc8b631d47745919f94cb9a1362c22e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:19.692972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad35b2-686a-42c8-84be-5f9982874525 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.692972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad35b2-686a-42c8-84be-5f9982874525 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.692972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad35b2-686a-42c8-84be-5f9982874525 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.692972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 76/149] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:19.699464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fd6b5-61ed-489d-a11a-fa75c5e60f0f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.699994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fd6b5-61ed-489d-a11a-fa75c5e60f0f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.700512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360fd6b5-61ed-489d-a11a-fa75c5e60f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.713407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:19.722015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-360fd6b5-61ed-489d-a11a-fa75c5e60f0f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:19.723612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-360fd6b5-61ed-489d-a11a-fa75c5e60f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:19.801153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 74/150] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:19.824067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad35b2-686a-42c8-84be-5f9982874525 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:19.845452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb0057-917b-463b-a7c5-2eeb09b6cf60 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.845749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb0057-917b-463b-a7c5-2eeb09b6cf60 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.846021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecb0057-917b-463b-a7c5-2eeb09b6cf60 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.847532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 75/151] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:19.853601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:19.854178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:19.854666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:19.873620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:19.875320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:19.948007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad35b2-686a-42c8-84be-5f9982874525 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:19.954429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 76/152] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:19.961713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:19.989825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.012498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.013576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.014012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.014423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.037948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:20.047120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:20.049745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:20.050710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:20.053643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 77/153] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:20.062510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.108503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:20.175787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb51cd6e-1948-4248-baf3-f2075c3bba76 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:20.190973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fc77bd-7888-448f-920c-501f5294b345 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:20.240924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.241511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.241861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.242071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.242837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:20.243145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:20.244578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:20.263309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 78/154] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:20.263655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.264344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.264685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.264971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.274658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.285792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:20.295043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:20.296355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:20.297876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 77/155] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:20.302249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.348191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:20.473581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded6378-95fd-4a94-8215-f3045c949b11 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:20.479020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da76f23-e2e8-4d7d-afe4-a6fb858cb77e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:20.532407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:20.540940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.542115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.542403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.543049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.543828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:20.544161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:20.545587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:20.548208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.549144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.549600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.550032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.551409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:20.553593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:20.555438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 79/156] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:20 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:20.563391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 78/157] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:20.565510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.565758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.738138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:20.744318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4916a283-e3f7-4217-b963-3fabeeb6cc88 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:20.787630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bb897-8c7a-4068-beaf-195eea122c53 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:20.820203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.820869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.821151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.821428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.855442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:20.856361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:20.856644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:20.856918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] PATH_INFO: `/v3/users/477b9e36f0344a75a2c77704966bc3a5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:20.860907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] RBAC: Authorizing `identity:get_user(user_id=477b9e36f0344a75a2c77704966bc3a5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:20.862768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:20.868819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 79/158] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:20 2022] GET /identity/v3/users/477b9e36f0344a75a2c77704966bc3a5 => generated 247 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:20.874833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:20.928459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:20.930293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:20.968739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:21.041306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7654-e789-4d97-bf30-8aa2698d10c3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:21.100973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:21.106737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 80/159] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:21.112710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:21.113744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:21.114296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:21.114784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:21.118906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:21.264208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:21.276545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:21.374178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f437f75a-f96b-40b5-9a8a-b08518895e47 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:21.424471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:21.448749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:21.449615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:21.449893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:21.450172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:21.451382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:21.461196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:21.463417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 81/160] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:21 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:21.472597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:21.523324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 80/161] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 649 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:11:21.531970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:21.652115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:21.742695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b4ca69-854c-4bd0-9e83-8ab5ce4f9b69 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:21.746250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce33b7-c06d-40c8-89f8-5351351cde35 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:21.815968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:21.817936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:21.819235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:21.819981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:21.825048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:21.827529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:21.830787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 81/162] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:21 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:21.837907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:21.845563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:21.846808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:21.847245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:21.847812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:21.849408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=30cadd9bcfd84eaba7b31000da7993e1) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:21.850052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:21.857028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4eb733-9656-46f0-af63-44fc3d1aefea None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:21.876229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 82/163] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:21 2022] GET /identity/v3/role_assignments?user.id=30cadd9bcfd84eaba7b31000da7993e1&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 608 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:21.881687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:21.938905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c285563-e82b-4bf7-9eb1-6d2732633a98 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:21.939898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c285563-e82b-4bf7-9eb1-6d2732633a98 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:21.940477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c285563-e82b-4bf7-9eb1-6d2732633a98 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:21.942130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 83/164] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:21.947762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e676161-8952-41a2-8be6-f1275980f56c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:21.948372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e676161-8952-41a2-8be6-f1275980f56c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:21.948845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e676161-8952-41a2-8be6-f1275980f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:21.980601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6e676161-8952-41a2-8be6-f1275980f56c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:21.983188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6e676161-8952-41a2-8be6-f1275980f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:22.058386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c3c05-27d7-46f7-8c94-6eab8d4ae183 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:22.138291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 84/165] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:22.139669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824dc31-5aff-4c43-9e56-0e62020ea419 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.139947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824dc31-5aff-4c43-9e56-0e62020ea419 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.140578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824dc31-5aff-4c43-9e56-0e62020ea419 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.148832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 85/166] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:22 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:22.155200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.155824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.157354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.184308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:22.185454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.185945 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.186484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.187778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=477b9e36f0344a75a2c77704966bc3a5) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:22.188349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:22.190126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:22.196305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:22.198006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:22.198948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 82/167] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:21 2022] GET /identity/v3/role_assignments?user.id=477b9e36f0344a75a2c77704966bc3a5&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:22.252112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:22.308804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:22.323541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 86/168] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:22.360611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:22.438175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:22.471560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d7ae3-00c7-41c3-b321-6cf2d0e3658d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:22.558692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:22.560328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.560627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.560908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.575879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a996598-fd34-4519-910f-bfa7cfbdb0cc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:22.582670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:22.592320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:22.593414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:22.594183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 83/169] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:22.600359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:22.645672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:22.647368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.647864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.648385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.675015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:22.683944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:22.684844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:22.685619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:22.687993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 87/170] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:22.694030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:22.782641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445fd67-9596-48c7-adca-59b212174a99 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:22.816232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:22.860625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:22.861312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.861594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.861870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.862633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:22.862939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:22.864372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d279b3-f8ab-4e4b-9ed1-26cf9ee6a170 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:22.881273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 84/171] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:22.882959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b84cc-c498-4f65-96a9-6e6bded6a0a6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.883256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b84cc-c498-4f65-96a9-6e6bded6a0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.883539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b84cc-c498-4f65-96a9-6e6bded6a0a6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.885025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 85/172] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:22.894432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.894720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.895188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.909765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd7ddd-0c95-45f8-b4c4-108d132778cb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:22.933120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:22.934332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:22.941936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:22.960618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:22.961579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:22.961864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:22.962139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:22.963167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:22.963474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:22.964922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:22.977431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 88/173] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:22.978303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.065918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 86/174] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 15 19:11:23.067016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.174259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbffc351-244b-45bd-8e53-494c986977e3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:23.178911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20cf26-2604-4c2a-a916-e1bea68252d0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:23.208462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:23.211318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:23.212548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.213129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.213609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.215063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:23.217110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:23.219028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 87/175] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:23.220119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.242167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:23.255495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:23.258216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.259021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.260487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] PATH_INFO: `/v3/users/30cadd9bcfd84eaba7b31000da7993e1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.262343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] RBAC: Authorizing `identity:get_user(user_id=30cadd9bcfd84eaba7b31000da7993e1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:23.265027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:23.267573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 89/176] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:22 2022] GET /identity/v3/users/30cadd9bcfd84eaba7b31000da7993e1 => generated 275 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:23.268673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.435587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23bf093-707c-4391-9d32-54667a1e9447 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:23.460562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c1f38-fe9c-430e-b083-4c48e491e067 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:23.506700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:23.507586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.507870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.508170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.527021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:23.527680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.527963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.528265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.529339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:23.549081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:23.550015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:23.550785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:23.551845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 88/177] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:23.553072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.610782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:23.632344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:23.633950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:23.636014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:23.686054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f38db93-4ff4-49bf-bb0c-ae4c594e53ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:23.727214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c880d1-0abb-42d6-8c44-32044023b532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:23.728483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c880d1-0abb-42d6-8c44-32044023b532 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.728889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c880d1-0abb-42d6-8c44-32044023b532 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.729175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c880d1-0abb-42d6-8c44-32044023b532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.758810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 90/178] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:23.760140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.817355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c880d1-0abb-42d6-8c44-32044023b532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:23.820306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c880d1-0abb-42d6-8c44-32044023b532 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:23.870368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:23.898944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e8378-7dd5-4c14-8333-c9c4f6af8838 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:23.903590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:23.944033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:23.945488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.945989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.946468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.947325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 89/179] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:23.947773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:23.948311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:23.948883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d00cc-8657-4261-a35e-383e83b04ec1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:23.949174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d00cc-8657-4261-a35e-383e83b04ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:23.949456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d00cc-8657-4261-a35e-383e83b04ec1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:23.950597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:23.950774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 90/180] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:23.955098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d00cc-8657-4261-a35e-383e83b04ec1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:23.962866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 91/181] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:23.963785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.061998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d00cc-8657-4261-a35e-383e83b04ec1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.074458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658fd78a-50b7-497b-9116-bf9044ffb98e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.096197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25952e9-7afa-4f2f-8b23-bc923b7997af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.097057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25952e9-7afa-4f2f-8b23-bc923b7997af None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.097340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25952e9-7afa-4f2f-8b23-bc923b7997af None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.097607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25952e9-7afa-4f2f-8b23-bc923b7997af None admin] PATH_INFO: `/v3/projects/895bb12f67af4e8490a762a628b8475e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.098783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25952e9-7afa-4f2f-8b23-bc923b7997af None admin] RBAC: Authorizing `identity:get_project(project_id=895bb12f67af4e8490a762a628b8475e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.100312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25952e9-7afa-4f2f-8b23-bc923b7997af None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.102020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 91/182] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/projects/895bb12f67af4e8490a762a628b8475e => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:24.103146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0839ec-117b-4d19-99ae-559382dd7954 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.103426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0839ec-117b-4d19-99ae-559382dd7954 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.103697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0839ec-117b-4d19-99ae-559382dd7954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.118020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.118884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.119169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.119448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.120356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1e0839ec-117b-4d19-99ae-559382dd7954 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:24.120669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.121541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1e0839ec-117b-4d19-99ae-559382dd7954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:24.122123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.124101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 92/183] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:23 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:24.124867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.139640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.152075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.189692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 92/184] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:24.190983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b41c5-ff26-4780-afdc-1b1c7bbbd2ce None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.191268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b41c5-ff26-4780-afdc-1b1c7bbbd2ce None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.191541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b41c5-ff26-4780-afdc-1b1c7bbbd2ce None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.192839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 93/185] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:24.197379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b41c5-ff26-4780-afdc-1b1c7bbbd2ce None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.232847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0a6ba-e74a-4780-81a3-81a343c2ef33 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.267291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96392b0-a52c-4417-b10f-480517ab56ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.268180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96392b0-a52c-4417-b10f-480517ab56ed None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.268465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96392b0-a52c-4417-b10f-480517ab56ed None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.268744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96392b0-a52c-4417-b10f-480517ab56ed None admin] PATH_INFO: `/v3/users/9c192523a59d410aafadaea708fabd52` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.269922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96392b0-a52c-4417-b10f-480517ab56ed None admin] RBAC: Authorizing `identity:get_user(user_id=9c192523a59d410aafadaea708fabd52)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.271886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96392b0-a52c-4417-b10f-480517ab56ed None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.273714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 93/186] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/users/9c192523a59d410aafadaea708fabd52 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:24.274779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887921b1-d7cf-4dcb-990f-bf650ab64c7b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.275065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887921b1-d7cf-4dcb-990f-bf650ab64c7b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.275339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887921b1-d7cf-4dcb-990f-bf650ab64c7b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.276588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 94/187] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:24.280338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887921b1-d7cf-4dcb-990f-bf650ab64c7b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.309595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b41c5-ff26-4780-afdc-1b1c7bbbd2ce None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.343336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.343985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.344301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.344579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.345410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=895bb12f67af4e8490a762a628b8475e, user.id=30cadd9bcfd84eaba7b31000da7993e1) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:24.345720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.347167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca34163-b4fe-4517-b9f7-c5cfc9a54428 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.353368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 94/188] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/role_assignments?user.id=30cadd9bcfd84eaba7b31000da7993e1&scope.project.id=895bb12f67af4e8490a762a628b8475e&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:24.357195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.357433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.357723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.358006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.379032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:24.380304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:24.383183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887921b1-d7cf-4dcb-990f-bf650ab64c7b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.409399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.423872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.424595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.424880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.425156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.426014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] RBAC: Adding query filter params (role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=30cadd9bcfd84eaba7b31000da7993e1) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:24.426325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.428400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.436216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 95/189] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/role_assignments?user.id=30cadd9bcfd84eaba7b31000da7993e1&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:24.437098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.473808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 95/190] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:24.475738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.571321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c4425-26f4-48f0-8127-00afdd40e704 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.617064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45955954-b711-45f5-951c-cda39f7fabc7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.618948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70361c8-baaa-44b8-b680-6d9be6b17ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.619772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70361c8-baaa-44b8-b680-6d9be6b17ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.620125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70361c8-baaa-44b8-b680-6d9be6b17ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.620480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70361c8-baaa-44b8-b680-6d9be6b17ca6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.634179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70361c8-baaa-44b8-b680-6d9be6b17ca6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e70361c8-baaa-44b8-b680-6d9be6b17ca6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:24.642336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:24.643197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:24.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:24.646783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 96/191] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:24.648068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.648465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.648742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.656143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.666816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.667582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.667871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.668174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.669333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:24.670688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:24.706519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.741971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 97/192] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:24.742744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.748727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.750483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.843601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b0c35-9e81-4502-9029-12e8d163598f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.854853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 96/193] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:24.855709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.878911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.879705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.879987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.880296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.882132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.883813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:24.885960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 98/194] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:24.886574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:24.893057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.914544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:24.955048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b839a5d-5a5d-406a-8a32-81f591fc9d82 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.989119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:24.990351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:24.991024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e7e85d-1fae-48ed-9f8f-9992365493d8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:24.991464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:24.991464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:24.992246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:24.992513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:24.993918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.002160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 97/195] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:25.002941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.025894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.026693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.026972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.027240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.037485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:25.045503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:25.046481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:25.047263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 99/196] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:25.048121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.107312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:25.109712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7952c8-927a-488d-9f49-a87e699cb488 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.132926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:25.145075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.145930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.146209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.146481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.148395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.149440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e984a00-072a-477d-bb23-b06537eec5de None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.149853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.151548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 98/197] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:25.152322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.187402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.188068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.188382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.188656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.189363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.190811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.209486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 100/198] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:25 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:25.211178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.275309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6bac0d-8100-4917-abf5-6a20b58e2522 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.313310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.314683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.315195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.315647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] PATH_INFO: `/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.321297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db7573e-8127-4c4f-a8de-80512d5e5596 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.333697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] RBAC: Authorizing `identity:get_user(user_id=c9f7a7407cca42a6b2ba02cb82bd7eed)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.335259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.337106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 99/199] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:25.337909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.345648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:25.366601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.367522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.367965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.368297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.369058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:25.369369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.370764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.385014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 101/200] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:25.385014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.402691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:25.469761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffada803-0cc5-4aba-8bb0-bf80c6d720c8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.510156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9fa9c-11f1-4530-897b-542154bdda54 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.521133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.521796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.522072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.522369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.523274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] RBAC: Adding query filter params (role.id=7bc8b631d47745919f94cb9a1362c22e, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9, user.id=9c192523a59d410aafadaea708fabd52) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:25.523583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.525242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.539480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 100/201] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/role_assignments?user.id=9c192523a59d410aafadaea708fabd52&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=7bc8b631d47745919f94cb9a1362c22e => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:25.539480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.564349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.565689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.565971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.566518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.651943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:25.692461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.694057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.701679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b47e0c-f00e-4303-9d56-4d4fd1c8fe13 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.733229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:25.779155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.780227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.780619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.781015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] PATH_INFO: `/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:25.782359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] RBAC: Authorizing `identity:get_user(user_id=c9f7a7407cca42a6b2ba02cb82bd7eed)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:25.784040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:25.786370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 101/202] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed => generated 283 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:25.794978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.897956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 102/203] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:25.905050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:25.913371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a0f63-e62b-429a-82ed-ca268b6d4683 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:25.964905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:25.965897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:25.966358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:25.966809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:26.021444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:26.023947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:26.051052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f00fe7-05b6-4a81-ac79-255823e815af None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:26.055923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:26.057681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:26.095170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:26.096550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:26.097260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:26.097824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:26.100216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:26.102421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:26.104724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 103/204] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:26.110045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:26.189329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 102/205] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:26.195605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:26.237904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4086f668-8e46-4a98-a94a-31a5c64856d3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:26.283413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:26.294047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:26.295045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:26.295339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:26.295622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:26.296807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=c9f7a7407cca42a6b2ba02cb82bd7eed) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:26.297187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:26.299335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:26.307495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140a8b7d-6a2a-4e7c-9e1b-707d4a2e8cc9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:26.307617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 104/206] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:26 2022] GET /identity/v3/role_assignments?user.id=c9f7a7407cca42a6b2ba02cb82bd7eed&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:26.352354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bdc7bb-ceb7-407c-a556-d7e43544bebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:26.353617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdc7bb-ceb7-407c-a556-d7e43544bebe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:26.354104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdc7bb-ceb7-407c-a556-d7e43544bebe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:26.354628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bdc7bb-ceb7-407c-a556-d7e43544bebe None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:26.356154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdc7bb-ceb7-407c-a556-d7e43544bebe None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:26.357927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bdc7bb-ceb7-407c-a556-d7e43544bebe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:26.360374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 103/207] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:26 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:26.369250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:26.387463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:26.492400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d999ca-319f-41d7-9412-ce194cfb0f3d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:26.528782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:26.529713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:26.530139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:26.530556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:26.531541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] RBAC: Adding query filter params (role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=c9f7a7407cca42a6b2ba02cb82bd7eed) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:26.532014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:26.533733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb27c87-691f-42d4-b7fd-66e611cc2726 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:26.540531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 105/208] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:26 2022] GET /identity/v3/role_assignments?user.id=c9f7a7407cca42a6b2ba02cb82bd7eed&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:27.464430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7914c0-d431-44d4-9caa-b06c8d1f33a4 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.464978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7914c0-d431-44d4-9caa-b06c8d1f33a4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.464978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7914c0-d431-44d4-9caa-b06c8d1f33a4 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.466621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 104/209] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:27.471614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.471997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.472196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.489091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:27.490282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:27.532455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca64930-711c-4058-a94f-02bedb8e33d0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.532759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca64930-711c-4058-a94f-02bedb8e33d0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.533007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca64930-711c-4058-a94f-02bedb8e33d0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.534372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 105/210] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:27.541660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.542039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.542417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.559939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:27.561256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:27.566456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 106/211] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:27.590473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:27.598604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:27.641690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 106/212] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:27.666663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:27.720638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa1ee-f016-40d7-849e-47bb8bef5279 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:27.758535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:27.782669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:27.783966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.784692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.785118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.791598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa151-dc95-4a92-896a-534ce9b1d805 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:27.800225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:27.812054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:27.813149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:27.814143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:27.816265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 107/213] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:27.824142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:27.837037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:27.838502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.838882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.839256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.841002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:27.843237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:27.849819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 107/214] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:27.850840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:27.910574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:27.950176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c37f5-852e-4acd-a9a0-961539ff2d72 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:27.975856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b00b33-7da2-4379-ba17-9e696279f295 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:27.987327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:27.988148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:27.988533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:27.988910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:27.989902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:27.990161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:27.992221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:28.006577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 108/215] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:28.009145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:28.028356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:28.029402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.030097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.030520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.031850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:28.033496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:28.064594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 108/216] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:27 2022] POST /identity/v3/users => generated 297 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:11:28.142004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8d5c3-db23-4a47-9f6c-7ba01ef2a340 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:28.173731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:28.184217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b4683-a5b3-4890-84d1-3e878f8e4bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:28.185466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4683-a5b3-4890-84d1-3e878f8e4bdb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.185985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4683-a5b3-4890-84d1-3e878f8e4bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.186465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4683-a5b3-4890-84d1-3e878f8e4bdb None admin] PATH_INFO: `/v3/users/16dec4ffad044070bb5849165e370c09` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.204016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b4683-a5b3-4890-84d1-3e878f8e4bdb None admin] RBAC: Authorizing `identity:get_user(user_id=16dec4ffad044070bb5849165e370c09)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:28.205836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b4683-a5b3-4890-84d1-3e878f8e4bdb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:28.212108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 109/217] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/users/16dec4ffad044070bb5849165e370c09 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:28.214356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:28.217440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:28.328466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e137d-b0c6-4c15-8b23-f840c3bde6d2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:28.364654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:28.365613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.365899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.366743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.376787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21cbab-d2c2-40a4-85a9-55e78388760d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.377070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21cbab-d2c2-40a4-85a9-55e78388760d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.377353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21cbab-d2c2-40a4-85a9-55e78388760d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.378867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 110/218] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:28.384024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.384346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.384624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.402714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:28.404169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:28.463930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:28.466886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:28.470507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:28.499976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 111/219] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:28.523587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:28.600203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 109/220] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:28.604487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:28.665085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8ab5e-87a7-4aa0-a710-f1fe732030d7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:28.687629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:28.712254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3361511-da81-4055-a4b0-d3d1c11e8ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:28.713061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3361511-da81-4055-a4b0-d3d1c11e8ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.713343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3361511-da81-4055-a4b0-d3d1c11e8ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.713620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3361511-da81-4055-a4b0-d3d1c11e8ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.725666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3361511-da81-4055-a4b0-d3d1c11e8ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:28.734173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4318fd32-6a5e-4604-99c5-5b7979c63133 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e3361511-da81-4055-a4b0-d3d1c11e8ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:28.734398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:28.735319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:28.736103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 112/221] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:28.737512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc618f1-d9f3-4e06-8cde-34b52050ae26 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.737512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc618f1-d9f3-4e06-8cde-34b52050ae26 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.737512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc618f1-d9f3-4e06-8cde-34b52050ae26 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.738794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 113/222] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:28.744872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc618f1-d9f3-4e06-8cde-34b52050ae26 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:28.747976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:28.777846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af31425-79af-4ea2-8254-d55c5b6a5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:28.778982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31425-79af-4ea2-8254-d55c5b6a5a79 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.779466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31425-79af-4ea2-8254-d55c5b6a5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.779959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af31425-79af-4ea2-8254-d55c5b6a5a79 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.781461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af31425-79af-4ea2-8254-d55c5b6a5a79 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:28.783166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af31425-79af-4ea2-8254-d55c5b6a5a79 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:28.785497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 110/223] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:28.791659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.792289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.792786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.810776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:28.812312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:28.871148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc618f1-d9f3-4e06-8cde-34b52050ae26 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:28.884685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 111/224] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:28.885913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:28.931154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:28.931840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:28.932624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:28.932911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:28.933684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:28.933997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:28.935776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:28.949240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 114/225] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:28.950358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.003223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.006476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.011734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967815b3-31e5-4b81-8a67-5e12ada64c2a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.051019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.052062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.052597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.053112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.054332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=16dec4ffad044070bb5849165e370c09) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:29.054975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.057089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.072357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 112/226] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/role_assignments?user.id=16dec4ffad044070bb5849165e370c09&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:29.073759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.075820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef5313-3069-4a91-b2eb-0f9c2ca060e2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.132256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.133615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.133903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.134177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.161072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:29.179902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:29.181052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:29.181830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 115/227] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:29.187151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.212716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc5a9f-e0c9-4b7f-bb15-ecdd8658911f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.251936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.253264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.253760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.254284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] PATH_INFO: `/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.255743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] RBAC: Authorizing `identity:get_user(user_id=c9f7a7407cca42a6b2ba02cb82bd7eed)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.258343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.260661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 113/228] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:29 2022] GET /identity/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:29.269305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.330517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.337137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.341389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f7f9b-637c-4fad-b6ec-7e94650a92d5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.382351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.384046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.384571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.385049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.386024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:29.386539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.389099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.401343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 116/229] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:29.407774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.511860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca198f15-9989-40f1-9ae8-7ac62c78b81f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.539157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0adebc8-0d1d-410e-916e-1598d8b80e87 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.567543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.568829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.570477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.570988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.572475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.574926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.576971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 114/230] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:29 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:29.582972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.629668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.630561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.630844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.631117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] PATH_INFO: `/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.633281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] RBAC: Authorizing `identity:get_user(user_id=c9f7a7407cca42a6b2ba02cb82bd7eed)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.634682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.644164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 117/231] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:29 2022] GET /identity/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:29.645087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.721738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.769527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.779232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e2549-328e-4cd5-92af-caba8f68d26e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.782004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3af8ae-5578-44b2-bca1-255a5368c9c9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.821528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.822390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.822673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.822946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.824149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.825750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.827459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 118/232] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:29 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:29.836933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:29.885413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b4130-e0a0-41e4-951f-b9fe0b4ab60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.886239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4130-e0a0-41e4-951f-b9fe0b4ab60c None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.886525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4130-e0a0-41e4-951f-b9fe0b4ab60c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.886802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b4130-e0a0-41e4-951f-b9fe0b4ab60c None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/c9f7a7407cca42a6b2ba02cb82bd7eed/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.888843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4130-e0a0-41e4-951f-b9fe0b4ab60c None admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a48b00224ac59fc18cab7ad35465, user_id=c9f7a7407cca42a6b2ba02cb82bd7eed, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:29.890295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2b4130-e0a0-41e4-951f-b9fe0b4ab60c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:29.924168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 115/233] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:29 2022] PUT /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/c9f7a7407cca42a6b2ba02cb82bd7eed/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:11:29.948594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:29.956182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df56ce-9079-485c-b160-a918fc420bd1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:29.995189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32114b-2bcb-4b15-9ba4-eb471d97fd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:29.996107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32114b-2bcb-4b15-9ba4-eb471d97fd87 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:29.996409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32114b-2bcb-4b15-9ba4-eb471d97fd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:29.996688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32114b-2bcb-4b15-9ba4-eb471d97fd87 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/c9f7a7407cca42a6b2ba02cb82bd7eed/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:29.999158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32114b-2bcb-4b15-9ba4-eb471d97fd87 None admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a48b00224ac59fc18cab7ad35465, user_id=c9f7a7407cca42a6b2ba02cb82bd7eed, role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:30.000652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32114b-2bcb-4b15-9ba4-eb471d97fd87 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:30.021514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 119/234] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:29 2022] PUT /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/c9f7a7407cca42a6b2ba02cb82bd7eed/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:30.603958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d9eb-6522-48ea-8e24-8f7e8dc98d96 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:30.604607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d9eb-6522-48ea-8e24-8f7e8dc98d96 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:30.604607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c2d9eb-6522-48ea-8e24-8f7e8dc98d96 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:30.606095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 116/235] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:30.626188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:30.626680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:30.626781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:30.649089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:30.650299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:30.710132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e43194-7a1a-4c65-a7ce-47e7e96c3b6e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:30.710561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e43194-7a1a-4c65-a7ce-47e7e96c3b6e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:30.710720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e43194-7a1a-4c65-a7ce-47e7e96c3b6e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:30.712057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 117/236] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:30.725212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:30.726172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:30.726459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:30.728395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 120/237] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:30.748302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:30.749998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:30.760370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:30.825161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 118/238] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:30.860883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:30.914139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:30.931828 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975600b7-bca0-4fca-905f-d8945fe0819a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:30.972813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:30.973917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:30.975193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:30.975703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:30.978870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:30.980562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:30.984406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 121/239] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:30.991556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:31.054952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ad966-1689-4402-9b99-33b0add1cd42 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:31.088494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:31.097115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:31.098129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.101337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.102012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.124294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:31.139373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:31.140304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:31.141073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:31.143446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 119/240] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:31.153324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:31.267615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a8d4c-b102-4304-afbe-a059186c0ca2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:31.304317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:31.305445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.305955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.306770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.307890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:31.308443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:31.310068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:31.320493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 120/241] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:31.326274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:31.330707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d02d2-aed8-42d9-99c8-6dccd19e3f16 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:31.360457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:31.455637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a190f99a-066b-4733-a4e7-79ae32a58b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:31.456266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f99a-066b-4733-a4e7-79ae32a58b7e None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.456549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f99a-066b-4733-a4e7-79ae32a58b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.456824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190f99a-066b-4733-a4e7-79ae32a58b7e None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.457547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190f99a-066b-4733-a4e7-79ae32a58b7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:31.459009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a190f99a-066b-4733-a4e7-79ae32a58b7e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:31.461634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92252674-7796-41a1-8d25-f796eaf3a8bf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:31.498438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:31.499671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.500024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.500671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] PATH_INFO: `/v3/users/00aac5c496d649eda4f56c57a62bc3c1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.515988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] RBAC: Authorizing `identity:get_user(user_id=00aac5c496d649eda4f56c57a62bc3c1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:31.517611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:31.519338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 121/242] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:31 2022] GET /identity/v3/users/00aac5c496d649eda4f56c57a62bc3c1 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:31.523664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 122/243] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:30 2022] POST /identity/v3/users => generated 293 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:31.524780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:31.602356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:31.637347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e642e7d-d645-4b6e-9945-5a937e6820f5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:31.649761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb00eb-29c8-436a-82d0-852ebfc3b97d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.650272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb00eb-29c8-436a-82d0-852ebfc3b97d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.650343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb00eb-29c8-436a-82d0-852ebfc3b97d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.651692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 123/244] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:31.661622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.661973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.662174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.674915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:31.675610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:31.675895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:31.676208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:31.709243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:31.711131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:31.718058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:31.750427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:31.752118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:31.844518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 124/245] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:31.846568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:31.867462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:31.876501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 122/246] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:11:31.881884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.002199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b829bfb-60f3-4e90-997a-ada168ef13ff None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.013556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7f670-b3cf-4974-92c2-a66e1ca66097 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.051138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.052022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.052387 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.052737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.055436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:32.057100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:32.060185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 123/247] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:31 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:32.065934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.066381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.066757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.067044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.067325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.079822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:32.088252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:32.097313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:32.098223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:32.098984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:32.102198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 125/248] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:32.108241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.122874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:32.202129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5f5940-85e4-433e-a00e-34b15216131e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.230744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11962a5-1413-4bf2-b4e6-bd533650990c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.246329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.247441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.248266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.249409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.250855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] RBAC: Adding query filter params (role.id=7bc8b631d47745919f94cb9a1362c22e, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=00aac5c496d649eda4f56c57a62bc3c1) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:32.251340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:32.253395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:32.261695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 124/249] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:32 2022] GET /identity/v3/role_assignments?user.id=00aac5c496d649eda4f56c57a62bc3c1&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=7bc8b631d47745919f94cb9a1362c22e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:32.273106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.274098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.274555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.275026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.275970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:32.276488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:32.278089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c8c46b-e09f-4912-a9b2-ee174c87cc1c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:32.289110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 126/250] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:32.295045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.405955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e9d9d-d736-45a2-8333-dad9fcbb65b6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.408146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:32.443798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.445544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.445997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.446412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] PATH_INFO: `/v3/users/16dec4ffad044070bb5849165e370c09` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.447817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] RBAC: Authorizing `identity:get_user(user_id=16dec4ffad044070bb5849165e370c09)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:32.449665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:32.451597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 125/251] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:32 2022] GET /identity/v3/users/16dec4ffad044070bb5849165e370c09 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:32.458124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90025a-0638-4712-8fa8-3295dc692c5d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.458124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90025a-0638-4712-8fa8-3295dc692c5d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.458331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90025a-0638-4712-8fa8-3295dc692c5d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.460772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 127/252] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:32.462217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.468073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a311bb-2077-4eca-b473-a6c7a6745b87 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.468432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a311bb-2077-4eca-b473-a6c7a6745b87 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.468720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a311bb-2077-4eca-b473-a6c7a6745b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.501279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-93a311bb-2077-4eca-b473-a6c7a6745b87 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:32.502528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-93a311bb-2077-4eca-b473-a6c7a6745b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:32.584725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0457a3e4-f6a7-4541-96b8-9fed0d0826b7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.624772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:32.625642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.626972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.627443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.628004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.629770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:32.632407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:32.634969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 126/253] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:32 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:32.640616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 128/254] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:32.641622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f343949-6111-46e4-ae90-34b8b9aef98f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.688219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f343949-6111-46e4-ae90-34b8b9aef98f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.688670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f343949-6111-46e4-ae90-34b8b9aef98f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.690215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 129/255] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:32.696232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.696797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.697257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.707823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:32.719173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:32.721405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:32.818447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d62687-f91d-41da-9b48-e39d05bbaff2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:32.857166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 130/256] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:32.857864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:32.891414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:32.892338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:32.892619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:32.892895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/16dec4ffad044070bb5849165e370c09/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:32.898124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a48b00224ac59fc18cab7ad35465, user_id=16dec4ffad044070bb5849165e370c09, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:32.899582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:32.942784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 127/257] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:32 2022] PUT /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/16dec4ffad044070bb5849165e370c09/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:32.947385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.093774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f616f4a7-27f9-461c-b1bc-81a9b535ee3a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.120473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:33.135786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.137060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.138774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.139194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.160018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:33.169732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:33.170831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:33.171763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:33.176261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:33.181012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909e268a-1fea-44bc-b970-e50d6054e797 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.183210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 131/258] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:33.190962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.284849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.285703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.285988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.286266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.311616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b766-2155-45a9-adba-d486d2e6ac00 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.321697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:33.330109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:33.331135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:33.332042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:33.337079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 128/259] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:33.347400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.352734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.353416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.353699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.353985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.354833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:33.355139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:33.356700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:33.365548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 132/260] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:33.377251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.440445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:33.525833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96fed7-b76f-4991-b9e7-ffe102052fed None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.553689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a9ef11-4fa1-4383-8831-058a7030b25b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.563710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.564616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.565045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.565472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.566388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:33.566839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:33.568414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:33.583763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 129/261] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:33.585520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.620508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:33.635038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.635905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.636175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.636458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] PATH_INFO: `/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.641044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] RBAC: Authorizing `identity:get_user(user_id=c9f7a7407cca42a6b2ba02cb82bd7eed)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:33.642529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:33.653526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 133/262] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:33 2022] GET /identity/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:33.669690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.775360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27a223-3e2a-4106-a093-5dc2029d9357 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.812501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d25c52-fd54-49dc-9fd6-76461bc462da None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.827973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:33.842279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.844035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.844683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.846340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] PATH_INFO: `/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.848443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] RBAC: Authorizing `identity:get_user(user_id=c9f7a7407cca42a6b2ba02cb82bd7eed)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:33.850984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:33.856494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 130/263] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:33 2022] GET /identity/v3/users/c9f7a7407cca42a6b2ba02cb82bd7eed => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:33.862067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:33.869659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:33.870624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:33.871102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:33.871591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:33.958587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:33.977682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596bc60-bdba-4979-99c1-054f83f0af1a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:33.979202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:33.980912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:34.019103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:34.020128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.020656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.021131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:34.124449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 134/264] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:11:34.133736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:34.135500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:34.137117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:34.184445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:34.245083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:34.278813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 131/265] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:34.284284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:34.315800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d4086-9244-4dda-832d-14e3ae453cd6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:34.397999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:34.398936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.399560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.400063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:34.403317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:34.406424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:34.406424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 135/266] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:34 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:34.413245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:34.502362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af22e3c-8443-45ef-b9db-dcc1bac567b4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:34.540632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:34.577221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea8bc72-6c8f-4f3b-aa7e-1870e3b7b96c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:34.596784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:34.597667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.597949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.598224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:34.599448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:34.601555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:34.603799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 132/267] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:34 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:34.621279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:34.623404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:34.624584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.625032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.625447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:34.626422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=c9f7a7407cca42a6b2ba02cb82bd7eed) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:34.626872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:34.628485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316b872a-ff3c-43a8-8f15-441fcc3ed343 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:34.634981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 136/268] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:34 2022] GET /identity/v3/role_assignments?user.id=c9f7a7407cca42a6b2ba02cb82bd7eed&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:34.717572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:34.780110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec13d4f-f96f-43d4-8422-e0c5b8f87e61 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:34.811232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:34.811898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.812204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.812484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:34.813330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] RBAC: Adding query filter params (role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=c9f7a7407cca42a6b2ba02cb82bd7eed) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:34.813637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:34.815100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025b9a8-87e3-4098-a9d3-2d75bac6d91a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:34.822031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 133/269] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:34 2022] GET /identity/v3/role_assignments?user.id=c9f7a7407cca42a6b2ba02cb82bd7eed&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:34.974471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f187baf-fdfa-4a72-a788-b0cb493a9079 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.975538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f187baf-fdfa-4a72-a788-b0cb493a9079 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.976013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f187baf-fdfa-4a72-a788-b0cb493a9079 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:34.978298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 137/270] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:34.983784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:34.984352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:34.984821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.009399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:35.010955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:35.064194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e482b0-932f-4a48-b53c-a65e7b6501e7 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.064794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e482b0-932f-4a48-b53c-a65e7b6501e7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.065266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e482b0-932f-4a48-b53c-a65e7b6501e7 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.066843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 138/271] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:35.073012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.073592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.074080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.096950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:35.098882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 134/272] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:35.099933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:35.139652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.196188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 139/273] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:35.222702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:35.226127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.269580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d213fb5a-134f-43db-9fe4-773b2be0431c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:35.311942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:35.313104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.313614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.314030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.315441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:35.317161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:35.318791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:35.321020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 135/274] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:35.327337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.351340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ae786-46cd-41c7-9b75-9e2c2936d5b6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:35.398505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:35.399998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.400524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.401556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.416418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:35.426456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:35.427342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:35.428167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:35.432228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 140/275] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:35.435150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.445905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b786f3-6347-453f-b231-4f9fa69ce35d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:35.464779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:35.483108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057cbcc-ce12-44d9-af8f-e29acdbb7706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:35.483778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057cbcc-ce12-44d9-af8f-e29acdbb7706 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.484061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057cbcc-ce12-44d9-af8f-e29acdbb7706 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.484365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057cbcc-ce12-44d9-af8f-e29acdbb7706 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.485246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057cbcc-ce12-44d9-af8f-e29acdbb7706 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:35.486835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e057cbcc-ce12-44d9-af8f-e29acdbb7706 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:35.513749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 136/276] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:11:35 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:11:35.558898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7a2bd-6d64-42b8-b4ad-09d99acea7ce None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.559247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7a2bd-6d64-42b8-b4ad-09d99acea7ce None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.559466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f7a2bd-6d64-42b8-b4ad-09d99acea7ce None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.561381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 137/277] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:35.566965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.567260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.567440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.572259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d3869-d55e-4211-b8b4-c9b962fa985a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:35.586727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:35.587937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:35.622718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:35.624422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.624914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.625464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.626432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:35.626938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:35.629953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:35.643996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 141/278] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:35.644368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:35.650005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.673113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 138/279] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:35.696197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.779476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d819131-8938-43e1-8c61-9231fb0194ae None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:35.822374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:35.823280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.823561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.824043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] PATH_INFO: `/v3/users/11b8572cc9814f88bc9cb8dfa9be86d7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.833929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6901098f-0f6a-4934-aa80-c9370f3844f5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:35.837814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:35.841452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] RBAC: Authorizing `identity:get_user(user_id=11b8572cc9814f88bc9cb8dfa9be86d7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:35.843735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:35.846151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 142/280] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/users/11b8572cc9814f88bc9cb8dfa9be86d7 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:35.853886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.885573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:35.886990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:35.887524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:35.887901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:35.901071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:35.908905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:35.909966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:35.910883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 139/281] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:35.918502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:35.928076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:35.979716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f4870-e4ad-436f-b2ab-5aaf5372bf6e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.020336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.021002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.021286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.021558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.023301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, user.id=11b8572cc9814f88bc9cb8dfa9be86d7, scope.system=all) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:36.023709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:36.025197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792cd6b3-faa2-47bc-a5aa-cb6fd06c4db8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:36.032804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 143/282] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/role_assignments?user.id=11b8572cc9814f88bc9cb8dfa9be86d7&scope.system=all&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:36.034377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d005b3-bba6-4e0d-8b9c-0ce8e0beec59 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.034905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d005b3-bba6-4e0d-8b9c-0ce8e0beec59 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.035387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d005b3-bba6-4e0d-8b9c-0ce8e0beec59 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.037056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 144/283] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:36.049683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.050310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.050817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.057837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35199a39-8919-442c-b241-1ff2c4c743f6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.096104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:36.097200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:36.097464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.098371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.098853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.099428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.100984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:36.101515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:36.103192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:36.113482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 140/284] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:36.122731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:36.177037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:36.206583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 145/285] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:36.257654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:36.288881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfa8ba8-5701-40d6-b5bf-74578441279a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.325339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.326461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.326883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.327295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] PATH_INFO: `/v3/users/00aac5c496d649eda4f56c57a62bc3c1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.328988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] RBAC: Authorizing `identity:get_user(user_id=00aac5c496d649eda4f56c57a62bc3c1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:36.330729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:36.333113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 141/286] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:36 2022] GET /identity/v3/users/00aac5c496d649eda4f56c57a62bc3c1 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:36.338764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:36.347944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:36.455221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:36.465505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698a015-239a-4e09-bbc3-73067401ccd0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.486331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57f4fe-e5a6-4256-86d3-697fafa89c43 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.526973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.528001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.528641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.529084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.540609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:36.549612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:36.550699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:36.551650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:36.554158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 146/287] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:36.559820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:36.585460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.587618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.587618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.587618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.594156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:36.595889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:36.599351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 142/288] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:36 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:36.609665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:36.742740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2fb15-c1b4-4a1d-a826-704c0dada4e1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.761024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:36.809378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff31c42-b951-4242-91a7-68980e2548d3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:36.816784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.817483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.817775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.818059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.819348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:36.819745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:36.821198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:36.839552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 147/289] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:36.845429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:36.849178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:36.851875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:36.852398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:36.852896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/00aac5c496d649eda4f56c57a62bc3c1/roles/7bc8b631d47745919f94cb9a1362c22e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:36.860893 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a48b00224ac59fc18cab7ad35465, user_id=00aac5c496d649eda4f56c57a62bc3c1, role_id=7bc8b631d47745919f94cb9a1362c22e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:36.862455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:36.888156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 143/290] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:11:36 2022] PUT /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/users/00aac5c496d649eda4f56c57a62bc3c1/roles/7bc8b631d47745919f94cb9a1362c22e => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:36.982015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9eb9a-b460-453e-a100-5f4bbd893d7f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:37.021130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:37.022713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:37.023005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:37.023330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] PATH_INFO: `/v3/users/16dec4ffad044070bb5849165e370c09` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:37.024696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] RBAC: Authorizing `identity:get_user(user_id=16dec4ffad044070bb5849165e370c09)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:37.026360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:37.028413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 148/291] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:36 2022] GET /identity/v3/users/16dec4ffad044070bb5849165e370c09 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:37.033969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:37.064510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:37.161843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f501e-ddaa-4d52-9a44-fb21ceb4e896 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:37.197923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:37.198683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:37.198973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:37.199314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:37.269498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:37.271039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:37.295272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:37.376696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 144/292] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:37.382634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:37.426484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:37.492879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5beea2-2cd7-49cc-8741-94c0bd610a0c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:37.547586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ce26d4-c7ea-41a9-82b7-5a439c6a59a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:37.548538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce26d4-c7ea-41a9-82b7-5a439c6a59a3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:37.548824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce26d4-c7ea-41a9-82b7-5a439c6a59a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:37.549104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ce26d4-c7ea-41a9-82b7-5a439c6a59a3 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:37.550435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce26d4-c7ea-41a9-82b7-5a439c6a59a3 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:37.551906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ce26d4-c7ea-41a9-82b7-5a439c6a59a3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:37.553753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 149/293] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:37 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:37.559774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:37.672446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73cc7b6-a1ae-4c81-9261-9008d0ee49c2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:37.708878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:37.709888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:37.710619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:37.710916 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:37.711205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:37.712054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=16dec4ffad044070bb5849165e370c09) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:37.712409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:37.714116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166de706-b454-4682-a5fb-f52f41060b89 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:37.720624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 145/294] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:37 2022] GET /identity/v3/role_assignments?user.id=16dec4ffad044070bb5849165e370c09&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:38.120258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271b1b7-47c0-450e-b990-9e76e174bdfd None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.120794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271b1b7-47c0-450e-b990-9e76e174bdfd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.120794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271b1b7-47c0-450e-b990-9e76e174bdfd None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.122439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 150/295] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:38.128231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.129079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.129559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.152810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:38.154008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:38.221322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca2d6a-d5ad-4065-8dac-4db10b1a8ac9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.221960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca2d6a-d5ad-4065-8dac-4db10b1a8ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.223765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ca2d6a-d5ad-4065-8dac-4db10b1a8ac9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.226232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 151/296] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:38.233770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.234309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.234940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.238555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 146/297] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:38.255136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:38.257251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:38.260701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.348267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 152/298] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:38.372558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.374783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56cb4c-6e95-4c77-a2ba-74f264371b86 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:38.395350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:38.415103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:38.421071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:38.422171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.422613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.423463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.438263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:38.449820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:38.450873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:38.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:38.453787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 147/299] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:38.459186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.494994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56858-5f5c-4944-93c5-b9301f9cf45e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:38.531373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:38.532434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.532852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.533284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.534655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:38.536270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:38.538959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 153/300] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:38.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.576467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41180d7b-27dd-4ea9-9c1f-b59d4308dccb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:38.612306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:38.612980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.613262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.613538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.614338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:38.614622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:38.616517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2094e-0ef5-46fb-89cb-3adddd094245 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:38.629729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 148/301] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:38.629729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19525-c3e4-4d41-a93f-745f05e20eaf None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.629729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19525-c3e4-4d41-a93f-745f05e20eaf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.629729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19525-c3e4-4d41-a93f-745f05e20eaf None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.629729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 149/302] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:38.633407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19525-c3e4-4d41-a93f-745f05e20eaf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.656445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52882852-7ac8-40b5-8706-3cc3df487ca2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:38.659585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:38.676841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:38.696904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9b5eb-d2b8-4ad9-92b2-19e36de009c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:38.697683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9b5eb-d2b8-4ad9-92b2-19e36de009c3 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.697968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9b5eb-d2b8-4ad9-92b2-19e36de009c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.698243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9b5eb-d2b8-4ad9-92b2-19e36de009c3 None admin] PATH_INFO: `/v3/groups` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.698938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9b5eb-d2b8-4ad9-92b2-19e36de009c3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:38.700630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc9b5eb-d2b8-4ad9-92b2-19e36de009c3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:38.718488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 154/303] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Jul 15 19:11:38 2022] POST /identity/v3/groups => generated 229 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:38.720295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.720898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.721286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.747782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19525-c3e4-4d41-a93f-745f05e20eaf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:38.748124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:38.749787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:38.796305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:38.797767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:38.799142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:38.799582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] PATH_INFO: `/v3/users/fa897567f495421088efec046245dab0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:38.817635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] RBAC: Authorizing `identity:get_user(user_id=fa897567f495421088efec046245dab0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:38.820504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:38.827793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 150/304] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:38 2022] GET /identity/v3/users/fa897567f495421088efec046245dab0 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:38.828969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.846245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 155/305] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:38.870098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:38.954319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:38.962188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde491d-c245-42db-bcd1-d973a580797b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.000423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.001146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:39.002289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.002767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.003259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.004329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] RBAC: Adding query filter params (role.id=7bc8b631d47745919f94cb9a1362c22e, user.id=fa897567f495421088efec046245dab0, scope.system=all) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:39.005113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:39.007970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:39.008361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a622bb2-ce20-410f-8253-fd2e0b5e05b2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.016005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 151/306] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Jul 15 19:11:38 2022] GET /identity/v3/role_assignments?user.id=fa897567f495421088efec046245dab0&scope.system=all&role.id=7bc8b631d47745919f94cb9a1362c22e => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:39.048418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.049233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.049519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.049794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.062611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:39.070661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:39.071596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:39.072406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:39.076422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 156/307] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:39.078544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:39.190960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc883cd-1a8b-4bf1-a436-f4e6af09abdd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.227969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.228590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.228873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.229144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.229897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:39.230204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:39.231589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a63254-64ba-41af-b5cd-9a377ddf3f9f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:39.245943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 152/308] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:39.247327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:39.359079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430df691-a592-485e-88d8-8eccfb3f8bf4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.385483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5df6e0-29ec-43b4-b71f-4e59d1ff8673 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.385777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5df6e0-29ec-43b4-b71f-4e59d1ff8673 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.386044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5df6e0-29ec-43b4-b71f-4e59d1ff8673 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.387633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 153/309] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:39.392666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.392950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.393226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.395247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.396142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.396433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.396712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] PATH_INFO: `/v3/users/11b8572cc9814f88bc9cb8dfa9be86d7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.398461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] RBAC: Authorizing `identity:get_user(user_id=11b8572cc9814f88bc9cb8dfa9be86d7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:39.399934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:39.402012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 157/310] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:39 2022] GET /identity/v3/users/11b8572cc9814f88bc9cb8dfa9be86d7 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:39.407673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:39.411740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:39.413868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:39.434479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:39.435180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:39.515057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 154/311] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:39.535328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf229314-5165-4d3b-8695-c282b3d211f4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.538561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:39.576709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.577952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.578441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.578950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] PATH_INFO: `/v3/system/users/11b8572cc9814f88bc9cb8dfa9be86d7/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.581205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=11b8572cc9814f88bc9cb8dfa9be86d7, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:39.583508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:39.598461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 158/312] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Jul 15 19:11:39 2022] PUT /identity/v3/system/users/11b8572cc9814f88bc9cb8dfa9be86d7/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:39.660419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff68a6-3554-4638-aef4-d5908070d57a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.700107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.700932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.701212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.701488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.714432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:39.722673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:39.723596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:39.724419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 155/313] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:39.731286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:39.824503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:39.840891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd041a9-ed71-4c1a-8490-4cd8d3a8ab24 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:39.875634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:39.876339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:39.876623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:39.876898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:39.877647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] RBAC: Adding query filter params (name=reader) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:39.877950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:39.879323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:39.888312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 159/314] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:39.894210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:39.910739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:40.001980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fc9f7-c675-4eb6-986f-11f2e4376128 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:40.035737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:40.036562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:40.036843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:40.037118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] PATH_INFO: `/v3/users/00aac5c496d649eda4f56c57a62bc3c1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:40.038886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] RBAC: Authorizing `identity:get_user(user_id=00aac5c496d649eda4f56c57a62bc3c1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:40.040378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:40.042303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 156/315] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:39 2022] GET /identity/v3/users/00aac5c496d649eda4f56c57a62bc3c1 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:40.047736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:40.148319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97693c5b-6150-4ad8-ad83-dc82dfc784df None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:40.182458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:40.183114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:40.183392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:40.183665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:40.202734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:40.262274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:40.263778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:40.369932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 160/316] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:40.375441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:40.462466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:40.482145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46badaf0-31e5-431e-a3c3-d76b54e89272 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:40.516772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaac72-a9e9-4061-843c-818cf4e20151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:40.517738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaac72-a9e9-4061-843c-818cf4e20151 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:40.518019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaac72-a9e9-4061-843c-818cf4e20151 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:40.518294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaac72-a9e9-4061-843c-818cf4e20151 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:40.519482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaac72-a9e9-4061-843c-818cf4e20151 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:40.520965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaac72-a9e9-4061-843c-818cf4e20151 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:40.523514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 157/317] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:40 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:40.529388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:40.580269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:40.642771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1858f-7412-47f9-8b71-65d57a8f86f0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:40.678660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:40.679323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:40.679610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:40.679889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:40.680753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] RBAC: Adding query filter params (role.id=7bc8b631d47745919f94cb9a1362c22e, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465, user.id=00aac5c496d649eda4f56c57a62bc3c1) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:40.681068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:40.682520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d02a62-31c2-48e9-9763-f7aae26e5735 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:40.688998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 161/318] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:11:40 2022] GET /identity/v3/role_assignments?user.id=00aac5c496d649eda4f56c57a62bc3c1&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=7bc8b631d47745919f94cb9a1362c22e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:40.876341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fc187-948d-43b7-9030-6c6f267208fc None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:40.876652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fc187-948d-43b7-9030-6c6f267208fc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:40.876957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fc187-948d-43b7-9030-6c6f267208fc None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:40.878547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 158/319] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:40.884039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:40.884378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:40.884628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:40.902819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:40.903929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:40.976442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 162/320] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:40.997975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fc187-948d-43b7-9030-6c6f267208fc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.099814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fc187-948d-43b7-9030-6c6f267208fc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:41.134402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efba63ad-a951-4cf3-a65f-cb42a111c2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:41.135182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba63ad-a951-4cf3-a65f-cb42a111c2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.135461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba63ad-a951-4cf3-a65f-cb42a111c2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.135735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efba63ad-a951-4cf3-a65f-cb42a111c2f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.136919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba63ad-a951-4cf3-a65f-cb42a111c2f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:41.138362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efba63ad-a951-4cf3-a65f-cb42a111c2f9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:41.140425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 159/321] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:41.145722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.191304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:41.200842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593a38-883c-4e60-9194-72287a4d5eb0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.201129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593a38-883c-4e60-9194-72287a4d5eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.201413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d593a38-883c-4e60-9194-72287a4d5eb0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.202930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 160/322] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:41.207725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.208010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.208335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.225684 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:41.226868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:41.237983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:41.252265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b30fe73-a728-4053-9c28-01dc8ced0b40 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:41.290225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:41.291346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.291634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.292197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] PATH_INFO: `/v3/groups` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.292898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:41.295066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:41.298291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 161/323] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:41.315423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 163/324] 10.222.0.17 () {60 vars in 1134 bytes} [Fri Jul 15 19:11:41 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:11:41.321089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.428726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528adb2-b68e-4e57-9d29-c7d2ec1ed7f7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:41.476153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:41.476863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.477142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.477411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.490748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:41.499115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:41.499984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:41.500805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 162/325] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:41.506402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.621918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840cbcd0-9a96-4ff0-9c86-5c63294840c0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:41.623473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:41.661022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:41.661749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.662030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.662301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.663783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:41.664117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:41.665560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bcb598-6840-4767-ab82-850f490cb31c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:41.674991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 164/326] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:41.680595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.690255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:41.703482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18793ad-28a9-4676-967e-8a95fc912146 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.703766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18793ad-28a9-4676-967e-8a95fc912146 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.704045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18793ad-28a9-4676-967e-8a95fc912146 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.705455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 165/327] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:41.714628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.714909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.715185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.733111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:41.734232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:41.823037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94d9ea0-3357-485f-a3b8-d5fee3cbf084 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:41.828127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 166/328] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:41.849731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.869577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:41.870833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:41.871316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:41.871788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] PATH_INFO: `/v3/users/fa897567f495421088efec046245dab0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:41.873274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] RBAC: Authorizing `identity:get_user(user_id=fa897567f495421088efec046245dab0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:41.875211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:41.877490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 163/329] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:41 2022] GET /identity/v3/users/fa897567f495421088efec046245dab0 => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:41.885167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:41.972191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:41.983081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dccb3-7fed-4b57-bebf-c30e7490e61b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:41.988535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:42.018664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88efe559-b7f0-4e75-aa1a-d5da33d69af2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:42.029111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:42.030128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:42.030558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:42.030973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:42.049445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:42.065237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:42.066147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:42.066926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:42.070836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 167/330] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:42.074793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:42.075833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:42.076876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:42.079936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:42.080252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] PATH_INFO: `/v3/system/users/fa897567f495421088efec046245dab0/roles/7bc8b631d47745919f94cb9a1362c22e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:42.082651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fa897567f495421088efec046245dab0, role_id=7bc8b631d47745919f94cb9a1362c22e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:42.083926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:42.098451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 164/331] 10.222.0.17 () {58 vars in 1334 bytes} [Fri Jul 15 19:11:41 2022] PUT /identity/v3/system/users/fa897567f495421088efec046245dab0/roles/7bc8b631d47745919f94cb9a1362c22e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:11:42.220813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e9463-3a76-4fc3-a289-13130068844c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:42.262378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:42.263411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:42.263891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:42.264425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:42.265405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:42.265930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:42.267631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:42.281725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 168/332] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:42.287051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:42.396145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136589-0b08-4b26-ba56-b32d503a7dd4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:42.434074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc507a47-6a07-4a29-9097-80e9971edbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:42.435749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc507a47-6a07-4a29-9097-80e9971edbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:42.436290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc507a47-6a07-4a29-9097-80e9971edbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:42.436789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc507a47-6a07-4a29-9097-80e9971edbb8 None admin] PATH_INFO: `/v3/users/11b8572cc9814f88bc9cb8dfa9be86d7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:42.438204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc507a47-6a07-4a29-9097-80e9971edbb8 None admin] RBAC: Authorizing `identity:get_user(user_id=11b8572cc9814f88bc9cb8dfa9be86d7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:42.439852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc507a47-6a07-4a29-9097-80e9971edbb8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:42.447026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 165/333] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:42 2022] GET /identity/v3/users/11b8572cc9814f88bc9cb8dfa9be86d7 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:42.448688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:42.469880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:42.560957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ad9bba-ebe0-4f1f-879c-8b837114ae25 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:42.609738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:42.610803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:42.611334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:42.611824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:42.612892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] RBAC: Adding query filter params (role.id=8dbe145322294429a2b040dd40e9a8e0, user.id=11b8572cc9814f88bc9cb8dfa9be86d7, scope.system=all) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:42.613404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:42.615037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c9322-95fd-46dd-b588-8489d20c9991 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:42.631943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 169/334] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Jul 15 19:11:42 2022] GET /identity/v3/role_assignments?user.id=11b8572cc9814f88bc9cb8dfa9be86d7&scope.system=all&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:44.217202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cc293-f7a0-4434-a6a3-3c42ac6a8186 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.218234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cc293-f7a0-4434-a6a3-3c42ac6a8186 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.218724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250cc293-f7a0-4434-a6a3-3c42ac6a8186 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.220366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 166/335] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:44.229521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.230410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.230889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.255832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:44.257970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:44.338602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 170/336] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:44.376474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cc293-f7a0-4434-a6a3-3c42ac6a8186 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:44.386587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:44.507876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250cc293-f7a0-4434-a6a3-3c42ac6a8186 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:44.548710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903ee19e-267e-4849-99a0-29cc6b7001b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:44.549940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ee19e-267e-4849-99a0-29cc6b7001b4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.550423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ee19e-267e-4849-99a0-29cc6b7001b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.550895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903ee19e-267e-4849-99a0-29cc6b7001b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.563223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903ee19e-267e-4849-99a0-29cc6b7001b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-903ee19e-267e-4849-99a0-29cc6b7001b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:44.572024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:44.572915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:44.573765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:44.578105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 167/337] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:44.581826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:44.589271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:44.629708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee39aba-a37d-43f7-866f-d7fcd09f9a42 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.630339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee39aba-a37d-43f7-866f-d7fcd09f9a42 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.630840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee39aba-a37d-43f7-866f-d7fcd09f9a42 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.633307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 168/338] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:44.638253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8238b-5344-4814-972a-c59a69cafcbd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.638779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8238b-5344-4814-972a-c59a69cafcbd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.639254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8238b-5344-4814-972a-c59a69cafcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.665162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-46f8238b-5344-4814-972a-c59a69cafcbd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:44.667182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-46f8238b-5344-4814-972a-c59a69cafcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:44.716211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71c4da-683a-4f4d-a12e-3294b4fb9e5f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:44.758935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:44.776914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:44.777599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.778113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.778455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 169/339] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:44.778862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.779791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa16f3a-c0b0-4158-af6c-d5cf521d9570 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.780099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa16f3a-c0b0-4158-af6c-d5cf521d9570 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.780381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa16f3a-c0b0-4158-af6c-d5cf521d9570 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.780604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:44.781100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:44.781699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 170/340] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:44.783013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:44.787011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cd599-a5f7-44c7-9bac-df45c60a1c29 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.787835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cd599-a5f7-44c7-9bac-df45c60a1c29 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.788161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345cd599-a5f7-44c7-9bac-df45c60a1c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.794687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 171/341] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:44.800065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:44.832614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-345cd599-a5f7-44c7-9bac-df45c60a1c29 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:44.834052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-345cd599-a5f7-44c7-9bac-df45c60a1c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:44.889679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:44.941630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dec74d-2feb-4260-bb2d-ff0be1349f41 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:44.955856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 171/342] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 15 19:11:44.958100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9631b0-75fc-469a-8932-68619cd30467 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.958732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9631b0-75fc-469a-8932-68619cd30467 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.960048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9631b0-75fc-469a-8932-68619cd30467 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:44.962643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 172/343] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:44.963619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9631b0-75fc-469a-8932-68619cd30467 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:44.995910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:44.997343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:44.997779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:44.998452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.097725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:45.101117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:45.112378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9631b0-75fc-469a-8932-68619cd30467 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.143003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:45.154742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf7f92-f6d7-46fa-86fc-012105ca7586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.155996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7f92-f6d7-46fa-86fc-012105ca7586 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.156526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7f92-f6d7-46fa-86fc-012105ca7586 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.157001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf7f92-f6d7-46fa-86fc-012105ca7586 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.176234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf7f92-f6d7-46fa-86fc-012105ca7586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0caf7f92-f6d7-46fa-86fc-012105ca7586 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:45.187024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:45.187944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:45.188776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:45.189468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:45.191056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 173/344] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:45.192539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.193006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.193463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.212239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:45.213736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:45.240510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 172/345] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:45.241564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.305382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 174/346] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:45.306835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.388713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0893262e-8736-4836-8c86-3a457a82415a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.435741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.437414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.437874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.438345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.453564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:45.461225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425592e8-ec52-4fa3-9e21-9b85b9e21f96 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:45.464994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:45.472831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 173/347] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:45.473622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.487729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:45.507498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:45.514570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.515520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.515999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.516769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.518644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:45.519180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:45.521060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:45.534252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 175/348] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:45.535815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.596641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b766c4-4b57-4164-9360-a2948a96cd22 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.643224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.644124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.644472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.644755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.645959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:45.647707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:45.649472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 174/349] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:45.650321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.674209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19256bcb-f51d-41f1-8105-b4adc3aea2ff None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.723267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.724129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.724422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.724700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.737738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:45.746868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:45.747770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 15 19:11:45.748589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:45.752276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 176/350] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:45.753218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.781538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:45.791126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7360cfa-32a6-404a-a1a5-97e9d4ba7feb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.799180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:45.832549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.833289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.833577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.833857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.835925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:45.836366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:45.837764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:45.851067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 175/351] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:45.852124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:45.871778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cedc33-ea4e-4b2d-95e9-f6c351803c2d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.910103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:45.910688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:45.910975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:45.911425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:45.975169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc1c7c-be11-490b-b809-0373a3173196 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:45.988997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:45.991143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.020808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.021790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.023228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.023228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.030358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.032220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.034096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 176/352] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:46.035121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.042026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.066639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.126476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 177/353] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:46.127618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.174606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b651fd-9c21-47af-b440-33c67a0ed262 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:46.212125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.212663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.212949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.213222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.213978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:46.214281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.216123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf291b-5b38-4057-a188-60e38a02a1ba None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.224641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 177/354] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:46.226519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19686a-a2ee-42ff-9271-0d3f584a5116 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.226805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19686a-a2ee-42ff-9271-0d3f584a5116 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.227080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c19686a-a2ee-42ff-9271-0d3f584a5116 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.228511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 178/355] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:46.229274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19686a-a2ee-42ff-9271-0d3f584a5116 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.256872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f835f3-84e6-40a3-98f8-1fc7fc34fc30 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:46.299903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a3f52f-bc16-423c-96f5-39a6b38c7881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.300594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3f52f-bc16-423c-96f5-39a6b38c7881 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.300878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3f52f-bc16-423c-96f5-39a6b38c7881 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.301154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a3f52f-bc16-423c-96f5-39a6b38c7881 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.312374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.334212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.356016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c19686a-a2ee-42ff-9271-0d3f584a5116 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:46.391569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3f52f-bc16-423c-96f5-39a6b38c7881 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.392256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.392916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.394051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.394051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.394233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a3f52f-bc16-423c-96f5-39a6b38c7881 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.394303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:46.394575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.396000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.402649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 179/356] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:46.403508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.528991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 178/357] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:46.530807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee06857-7926-4456-8dbe-e04c6cf440f2 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.531134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee06857-7926-4456-8dbe-e04c6cf440f2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.531382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee06857-7926-4456-8dbe-e04c6cf440f2 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.532711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 179/358] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:46.533556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee06857-7926-4456-8dbe-e04c6cf440f2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.536609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ff64f-95bb-43bc-a927-e5f411b8e9e4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:46.581033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f74554-d6c4-4a49-9179-4735bb42b6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.582221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f74554-d6c4-4a49-9179-4735bb42b6cb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.582669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f74554-d6c4-4a49-9179-4735bb42b6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.583099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f74554-d6c4-4a49-9179-4735bb42b6cb None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.585248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f74554-d6c4-4a49-9179-4735bb42b6cb None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.586932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f74554-d6c4-4a49-9179-4735bb42b6cb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.588689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 180/359] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:46.589262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.589874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.590254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.590819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.609815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:46.611009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:46.622080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.663490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee06857-7926-4456-8dbe-e04c6cf440f2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:46.701597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 181/360] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:46.702720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.715906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85b837-7441-4afa-8138-ba7349a93d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.717080 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85b837-7441-4afa-8138-ba7349a93d3b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.717473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85b837-7441-4afa-8138-ba7349a93d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.717846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85b837-7441-4afa-8138-ba7349a93d3b None admin] PATH_INFO: `/v3/users/fa897567f495421088efec046245dab0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.719431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85b837-7441-4afa-8138-ba7349a93d3b None admin] RBAC: Authorizing `identity:get_user(user_id=fa897567f495421088efec046245dab0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.721123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85b837-7441-4afa-8138-ba7349a93d3b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.723353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 180/361] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/users/fa897567f495421088efec046245dab0 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:46.724363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.724814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.725220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.750000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:46.751186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:46.837618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69002964-ef0a-48e3-a647-dcec4c7c7c9c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:46.849776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 181/362] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:46.851166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.884827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:46.886052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:46.886330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:46.886610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:46.888971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:46.890559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:46.893118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.893501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 182/363] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:46.894367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:46.914945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:46.984980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da4234-98f0-416f-a90d-773afdd8e81c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.020708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877425d8-4aa2-462e-9a1c-01d72e5ff3c6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.033079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.035416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.035416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.035416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.036399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.038908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.040604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 182/364] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.041450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.064644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.065575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.065862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.066969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.067822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] RBAC: Adding query filter params (role.id=7bc8b631d47745919f94cb9a1362c22e, user.id=fa897567f495421088efec046245dab0, scope.system=all) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:47.068459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.070121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.088255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 183/365] 10.222.0.17 () {56 vars in 1308 bytes} [Fri Jul 15 19:11:46 2022] GET /identity/v3/role_assignments?user.id=fa897567f495421088efec046245dab0&scope.system=all&role.id=7bc8b631d47745919f94cb9a1362c22e => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.089056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.181739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:47.201044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c44c1-b57d-4d3a-8fa5-6361d9bd3239 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.204659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:47.220255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f65717-7165-4a7d-868c-0e44370bbb14 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.253208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.255743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.255743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.255743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.272239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.273557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:47.273858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.274330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.274808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:47.282008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:47.282904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:47.283673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:47.288245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 183/366] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:47.289193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.291231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:47.301728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:47.302630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:47.304205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:47.307488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 184/367] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:47.308556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.414940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785c957-b1c1-462a-b802-9d3257a3d0d5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.442071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e56066-dfbb-4f6a-8c01-19a9d15e37a0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.463493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.463493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.463774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.463997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.467752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.469718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.471867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 184/368] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.472469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.480912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:47.482867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.483533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.484064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.484302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.485132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:47.485808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.487903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.495265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 185/369] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.496130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.510958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:47.612119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68746732-5f2f-470a-911e-abac5fc08333 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.663074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.663757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.663955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff440a7-7c14-43dd-96cd-f076dd3839dc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.664059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.664349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.665174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:47.665421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.667096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.677714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 185/370] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.679107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.711742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.713113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.713871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.714345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.715478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:47.716317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.718163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.733174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 186/371] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.734033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.799621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:47.809760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bbd59d-2460-497f-b446-16c267511d3a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.828508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:47.854700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.855946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.856428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.856927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:47.857816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:47.858332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:47.859894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:47.869229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 186/372] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:47.870412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:47.882443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee7b5e-2456-4de5-a8cd-b499e7857a33 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:47.962286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:47.962883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:47.963164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:47.963438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.075680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e51165-0063-4e7f-8585-fb488d722d1c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.100328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.101811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.132490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:48.133243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.133525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.133801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.178610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:48.181603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:48.207300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.209690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.273878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 187/373] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:48.280170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:48.344752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 187/374] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:48.350542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:48.401782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0061ea48-244e-44c3-a493-f930fe2fbc1d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.445793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:48.446674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.446957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.447232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.447437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:48.448466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.449922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.451912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 188/375] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:48 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:48.458413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:48.502299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:48.511631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cad75a-9700-4127-aaeb-add7a8efdd75 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.559126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:48.560324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.560611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.560885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] PATH_INFO: `/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.565595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] RBAC: Authorizing `identity:get_project(project_id=98ca6d3d93034fc49fb3c9265ccf0dcb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.567358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.570095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 188/376] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:48 2022] GET /identity/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb => generated 344 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:48.578075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:48.644921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffffd7f-a7f0-4eb4-a6f1-e50ab535cba2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.707365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:48.708250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.708535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.708809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.711114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.712610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.720238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 189/377] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:48 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:48.721095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:48.767656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecae22a-70e2-40ea-9ca7-91311a3bacc1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.808892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:48.813431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06deec82-4897-4cd6-824c-afb66ec63200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:48.814676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deec82-4897-4cd6-824c-afb66ec63200 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.815284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deec82-4897-4cd6-824c-afb66ec63200 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.815748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06deec82-4897-4cd6-824c-afb66ec63200 None admin] PATH_INFO: `/v3/groups/8712bca585aa41f9803d8411a4027b50` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.821754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:48.824269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06deec82-4897-4cd6-824c-afb66ec63200 None admin] RBAC: Authorizing `identity:get_group(group_id=8712bca585aa41f9803d8411a4027b50)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.826176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06deec82-4897-4cd6-824c-afb66ec63200 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.829216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 189/378] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:48 2022] GET /identity/v3/groups/8712bca585aa41f9803d8411a4027b50 => generated 229 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:48.830646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104da26-76d2-4f9b-b3ce-0024b9bcaf34 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.831119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104da26-76d2-4f9b-b3ce-0024b9bcaf34 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.831634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104da26-76d2-4f9b-b3ce-0024b9bcaf34 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.831793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259f33f-1eb4-4135-95e8-3c22fd7f6044 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.837685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 190/379] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:48.840591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104da26-76d2-4f9b-b3ce-0024b9bcaf34 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:48.870429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:48.871080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.871535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.871961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.873019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:48.873482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:48.875329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c333f4-a7be-4ade-8fb5-d9358c4e1258 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:48.881867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 190/380] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:48 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:48.883300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:48.883782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:48.885001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:48.902614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:48.904206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:48.967556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104da26-76d2-4f9b-b3ce-0024b9bcaf34 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:48.980458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 191/381] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:49.010711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:49.011397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:49.011681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:49.011957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:49.012900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] RBAC: Adding query filter params (group.id=8712bca585aa41f9803d8411a4027b50, role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=98ca6d3d93034fc49fb3c9265ccf0dcb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:49.013207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:49.014901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:49.016928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:49.023083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 191/382] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:48 2022] GET /identity/v3/role_assignments?group.id=8712bca585aa41f9803d8411a4027b50&scope.project.id=98ca6d3d93034fc49fb3c9265ccf0dcb&role.id=56597e05853f4660b09a96c651dd0c7a => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:49.122006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:49.143147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a2b74-8ad2-43ce-b281-ca63d76e6698 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:49.185571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:49.186446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:49.186730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:49.186998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:49.199652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:49.207926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:49.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:49.210378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 192/383] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:49.215874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:49.278470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:49.327332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f330a679-9b31-4c48-a667-8b395d175c1d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:49.363285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:49.363962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:49.364273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:49.364547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:49.365395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:49.365699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:49.367194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:49.376572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 192/384] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:49.382728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:49.517629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce06a0f-9b46-4267-9c5c-d759295b7066 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:49.558947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:49.560217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:49.560720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:49.561203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:49.562606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:49.564314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:49.566486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 193/385] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:49 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:49.571987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:49.710289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b5d8a-6805-4623-bf71-43b176290c8c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:49.727632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:49.746655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39f394a-be61-42b5-b8a6-ac5b738474af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:49.747525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f394a-be61-42b5-b8a6-ac5b738474af None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:49.747807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f394a-be61-42b5-b8a6-ac5b738474af None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:49.748091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39f394a-be61-42b5-b8a6-ac5b738474af None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:49.749555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f394a-be61-42b5-b8a6-ac5b738474af None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:49.751009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39f394a-be61-42b5-b8a6-ac5b738474af None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:49.753079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 193/386] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:49 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:49.758616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:49.797238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:49.932065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0180532a-ef97-45db-9fec-43adf458c440 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:49.978685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fb181-3402-4350-8bfb-020096493fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:49.979499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb181-3402-4350-8bfb-020096493fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:49.979780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb181-3402-4350-8bfb-020096493fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:49.980057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb181-3402-4350-8bfb-020096493fe4 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/groups/c643507620ce49438c8f34ed271f9f16/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:49.982368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fb181-3402-4350-8bfb-020096493fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9, group_id=c643507620ce49438c8f34ed271f9f16, role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:49.983818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fb181-3402-4350-8bfb-020096493fe4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:50.017670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 194/387] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Jul 15 19:11:49 2022] PUT /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/groups/c643507620ce49438c8f34ed271f9f16/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:11:50.141603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195b8b-a169-4cd9-832b-ee71e7badded None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.141961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195b8b-a169-4cd9-832b-ee71e7badded None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.142414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195b8b-a169-4cd9-832b-ee71e7badded None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.144397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 194/388] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:50.150621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.150913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.151367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.176622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:50.178441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:50.254036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 195/389] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:50.276935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195b8b-a169-4cd9-832b-ee71e7badded None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:50.392848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195b8b-a169-4cd9-832b-ee71e7badded None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:50.438831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f598ed8-05eb-4488-8668-f86b978a91ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:50.439638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f598ed8-05eb-4488-8668-f86b978a91ce None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.439922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f598ed8-05eb-4488-8668-f86b978a91ce None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.440227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f598ed8-05eb-4488-8668-f86b978a91ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.456728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f598ed8-05eb-4488-8668-f86b978a91ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5f598ed8-05eb-4488-8668-f86b978a91ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:50.464764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:50.465629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:50.466384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:50.469792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 195/390] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:50.472403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:50.488529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:50.570085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f38576-f726-450d-974a-71a509f901b7 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.570710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f38576-f726-450d-974a-71a509f901b7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.571205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f38576-f726-450d-974a-71a509f901b7 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.573161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 196/391] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:50.580119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.580119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.580119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.606740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:50.607922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:50.628901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7487143-e7a1-4a41-8e64-c002259f643d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:50.670176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:50.700905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:50.701288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 197/392] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:50.701451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.701739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.702013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.702769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:50.703074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:50.708110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:50.723633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 196/393] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:50.729722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:50.736210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:50.847620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82161da2-19cf-4af9-86ab-6d670d6dfec5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:50.885230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:50.886495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:50.886995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:50.887474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:50.888894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:50.890636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:50.892857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 198/394] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:50 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:50.898579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:50.916448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:50.972570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:50.976189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fcfeb3-14c2-4d6f-9cc9-80c15087bc9c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.036183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.036994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.037749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.038032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.058547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:51.073212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:51.074117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:51.074922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:51.075525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 197/395] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:51.080141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:51.082138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337f5ee-11ef-41f9-9317-de9620f53c1b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.139519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.140725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.141007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.141284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.144569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:51.146021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:51.152075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 199/396] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:50 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:51.155510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:51.272481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29ef2c-fc79-4f43-832a-e05a9f3adb02 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.326516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:51.339458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9868e2-f4ec-4d64-9dfd-eea05f7d04df None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.343669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.345194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.345782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.346259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.347170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:51.350473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:51.352075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:51.360449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:51.371816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 198/397] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:51.377227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:51.408425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9ca34-6531-4685-9327-d912b1b6cf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.409692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9ca34-6531-4685-9327-d912b1b6cf81 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.410189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9ca34-6531-4685-9327-d912b1b6cf81 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.410677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9ca34-6531-4685-9327-d912b1b6cf81 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/groups/c643507620ce49438c8f34ed271f9f16/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.413004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9ca34-6531-4685-9327-d912b1b6cf81 None admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a48b00224ac59fc18cab7ad35465, group_id=c643507620ce49438c8f34ed271f9f16, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:51.414817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d9ca34-6531-4685-9327-d912b1b6cf81 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:51.435899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 200/398] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Jul 15 19:11:51 2022] PUT /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/groups/c643507620ce49438c8f34ed271f9f16/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:51.517134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4812-cb2a-4ad1-9abc-0fa2f90e9c16 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.517458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4812-cb2a-4ad1-9abc-0fa2f90e9c16 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.517735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4812-cb2a-4ad1-9abc-0fa2f90e9c16 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.519526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 201/399] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:11:51.525578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.525930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.526146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.549711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:51.551732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:51.555711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315c2b1-6332-489a-993a-5c7319f03262 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.628410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.629570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.630489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.631028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.633310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:51.634921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:51.637132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 199/400] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:51 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:51.643284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:51.660933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 202/401] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:51.707064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:51.732126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:51.833040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:51.836973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a0dbb5-321a-4022-9bd0-21f2202ecc9c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.846246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb10c55-c4a0-4779-acb4-ddce86bfbac9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:51.895538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.896986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.897291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.897808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.899088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:51.900577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:51.902612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 200/402] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:51 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:51.905513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:51.906338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:51.906616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:51.906960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:51.908305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:51.935995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:51.944027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:51.944892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:51.945682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 203/403] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:51.957183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.045408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50eb4629-a9eb-49dc-a489-fd52b203a736 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.069018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:52.077745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a151b4c-8192-4c21-a0f9-41c20e5f9729 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.079749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d02e4-7b7f-4104-9ced-5e390c2e46aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.081156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d02e4-7b7f-4104-9ced-5e390c2e46aa None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.081981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d02e4-7b7f-4104-9ced-5e390c2e46aa None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.082457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d02e4-7b7f-4104-9ced-5e390c2e46aa None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/groups/c643507620ce49438c8f34ed271f9f16/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.087718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d02e4-7b7f-4104-9ced-5e390c2e46aa None admin] RBAC: Authorizing `identity:create_grant(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9, group_id=c643507620ce49438c8f34ed271f9f16, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:52.088959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d02e4-7b7f-4104-9ced-5e390c2e46aa None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:52.105799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 201/404] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Jul 15 19:11:51 2022] PUT /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9/groups/c643507620ce49438c8f34ed271f9f16/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:52.108051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9eff-0390-43e4-ad41-6fec2002658c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.108611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9eff-0390-43e4-ad41-6fec2002658c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.109120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9eff-0390-43e4-ad41-6fec2002658c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.115343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 202/405] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:52.117109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.117401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.117683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.137389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.138464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.138959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.139427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.140492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:52.141158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:52.143027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:52.151132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:52.153715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:52.167805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 204/406] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:52.171137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.221848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:52.285184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 203/407] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:52.313945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.329299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c781c0d-4e6e-4dc3-b6ce-152e220b78ea None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.385726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.386618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.386908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.387188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.388432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:52.389898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:52.396132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 205/408] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:52 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:52.401021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.438741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94d89a-d914-41c8-ad31-cbb1bde12c9d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.456420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:52.482618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.483395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.483731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.484058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.497667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:52.506950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:52.507858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:52.508685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 204/409] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:52.514867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.592923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:52.608152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce0890a-9164-4a8f-9b47-6a66de3eb43a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.647262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863ab5c-7119-4697-8601-6cf653774435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.648533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863ab5c-7119-4697-8601-6cf653774435 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.648963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863ab5c-7119-4697-8601-6cf653774435 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.649328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863ab5c-7119-4697-8601-6cf653774435 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.650639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863ab5c-7119-4697-8601-6cf653774435 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:52.651453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73fe4b4-a21c-484d-b236-c6180ce176e2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.653880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863ab5c-7119-4697-8601-6cf653774435 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:52.656066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 206/410] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:52 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:52.657425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037c54-e129-4c71-90fb-1398dd2c1d82 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.657810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037c54-e129-4c71-90fb-1398dd2c1d82 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.658187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe037c54-e129-4c71-90fb-1398dd2c1d82 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.659751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 207/411] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:52.669023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037c54-e129-4c71-90fb-1398dd2c1d82 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.689603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.690640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.691078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.691493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.692462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:52.692934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:52.694553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb99ea-9f78-4bfd-ac74-1bf163783f21 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:52.704303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 205/412] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:52.705960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.706453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.706942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.734022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:52.735811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:52.775794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:52.812176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe037c54-e129-4c71-90fb-1398dd2c1d82 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:52.824901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 206/413] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:52.826309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.849585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:52.850480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:52.850767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:52.851049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/groups/c643507620ce49438c8f34ed271f9f16/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:52.853319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] RBAC: Authorizing `identity:create_grant(project_id=ddf9a48b00224ac59fc18cab7ad35465, group_id=c643507620ce49438c8f34ed271f9f16, role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:52.855151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:52.873235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 208/414] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Jul 15 19:11:52 2022] PUT /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465/groups/c643507620ce49438c8f34ed271f9f16/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:52.876936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:52.901413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:52.969831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ee406-19f9-4a1f-902d-4e967dd0a134 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:53.025022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:53.026985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.027474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.027964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] PATH_INFO: `/v3/groups/8712bca585aa41f9803d8411a4027b50` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.029779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] RBAC: Authorizing `identity:get_group(group_id=8712bca585aa41f9803d8411a4027b50)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:53.031811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:53.034241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 207/415] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:52 2022] GET /identity/v3/groups/8712bca585aa41f9803d8411a4027b50 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:53.041129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:53.085405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:53.095922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3656d85-d90e-491d-ad56-f39e39d8c00d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:53.139784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:53.140724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.141373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.141713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.156754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:53.168379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:53.169269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:53.170108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 209/416] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:53.172876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c94764-beb3-420e-b641-35880e0fd928 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:53.176770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:53.216565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:53.218302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.218788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.219280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] PATH_INFO: `/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.221639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] RBAC: Authorizing `identity:get_project(project_id=98ca6d3d93034fc49fb3c9265ccf0dcb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:53.223329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:53.226148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 208/417] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:53 2022] GET /identity/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb => generated 344 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:53.232454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:53.301851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:53.344122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb963262-dd9a-4feb-87ad-5431c12ba94a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:53.411206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:53.411928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.412697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.412984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.413753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:53.414064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:53.415484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:53.437816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 210/418] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:53.442882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:53.448851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d70327-3cf3-4822-b690-53835e5872e2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:53.452378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:53.520031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d109c-f8c9-43f0-95a9-00e1ad31e497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:53.521497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d109c-f8c9-43f0-95a9-00e1ad31e497 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.522157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d109c-f8c9-43f0-95a9-00e1ad31e497 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.522572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d109c-f8c9-43f0-95a9-00e1ad31e497 None admin] PATH_INFO: `/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb/groups/8712bca585aa41f9803d8411a4027b50/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.525395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d109c-f8c9-43f0-95a9-00e1ad31e497 None admin] RBAC: Authorizing `identity:create_grant(project_id=98ca6d3d93034fc49fb3c9265ccf0dcb, group_id=8712bca585aa41f9803d8411a4027b50, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:53.526876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d109c-f8c9-43f0-95a9-00e1ad31e497 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:53.551124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 209/419] 10.222.0.17 () {58 vars in 1442 bytes} [Fri Jul 15 19:11:53 2022] PUT /identity/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb/groups/8712bca585aa41f9803d8411a4027b50/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:11:53.616364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970f189d-09ca-4750-bd6d-1b9e0ecf4abd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:53.666317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:53.666998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.667285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.667565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.688504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:53.751602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:53.753111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:53.867926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 211/420] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:53.871341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b775ac-59cd-4852-beaf-06d68a1099b2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.871797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b775ac-59cd-4852-beaf-06d68a1099b2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.871970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b775ac-59cd-4852-beaf-06d68a1099b2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.873382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 210/421] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:53.874595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:53.882535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:53.883116 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:53.883450 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:53.913472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:53.914759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:53.918346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:53.983758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99d96c-b0fa-4bbc-9ab3-3c31ccc971dd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.046902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.047776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.048057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.048357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.050200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:54.051658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:54.053702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 212/422] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:53 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:54.061054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.065880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 211/423] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:54.114332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.217900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb387b1-40a6-46a4-8022-1b6695e15621 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.231656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:54.264553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:54.269656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.270600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.273305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae8eaa-b590-4ff8-856a-bd303b5ecc40 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.276339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.276660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.278000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:54.279486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:54.281346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 213/424] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:54.293079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.335415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.336235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.336513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.336784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.356232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:54.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:54.375535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:54.376340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:54.377860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 212/425] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:54.382520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.484327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc3120-b8a3-4213-ad16-87d9f4e4035c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.522983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.524134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.524639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.525060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.526163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:54.526482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:54.528034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28360302-e2ca-4cc8-b5d4-ea0a1185d05b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:54.530296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf0a6b-5372-49a7-8416-9154be6be6d1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.534328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 214/426] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 611 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:54.540120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2444675-8de7-4924-ab39-c3174b763788 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.540120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2444675-8de7-4924-ab39-c3174b763788 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.540120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2444675-8de7-4924-ab39-c3174b763788 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.542658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 215/427] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:54.546491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.546491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.546491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.563038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:54.564387 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:54.599144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.600898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.601417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.601831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.602755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:54.603201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:54.603749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:54.605931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:54.617817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 213/428] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:54.625262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.650624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 216/429] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:54.656498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:54.674782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.794847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8c2fa-3a5b-48f8-a7a8-413d8934f1f6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.832514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.833319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.833599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.834039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.845548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:54.849866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366475f-879e-4be5-8a5e-4a11007988e9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:54.854509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:54.855364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:54.856266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 217/430] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:54.863374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:54.898267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:54.900339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:54.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:54.900922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:54.904065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:54.987058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494c0ed-cf9e-4060-9f5a-5f2234f2c3fc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:54.997118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:54.999533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.020798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.031817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.032324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.032611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.032940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.033725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:55.034036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.035550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.051294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 218/431] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:55.057292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.143554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 214/432] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:11:55.153097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.169451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.181092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f067-2ac2-4bfe-81ec-cfce41c61c0d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.223711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eccf6a8-c501-49bd-a74a-9fd03aa4447b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.224977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf6a8-c501-49bd-a74a-9fd03aa4447b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.225476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf6a8-c501-49bd-a74a-9fd03aa4447b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.225758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eccf6a8-c501-49bd-a74a-9fd03aa4447b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.270232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00303e-0a6a-49df-9803-015374f1cb85 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.285693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.301898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccf6a8-c501-49bd-a74a-9fd03aa4447b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.304263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eccf6a8-c501-49bd-a74a-9fd03aa4447b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.306884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd68e9c-56d2-4d3f-b97b-db406c55dd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.307775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd68e9c-56d2-4d3f-b97b-db406c55dd5d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.308643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd68e9c-56d2-4d3f-b97b-db406c55dd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.308922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd68e9c-56d2-4d3f-b97b-db406c55dd5d None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.310099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd68e9c-56d2-4d3f-b97b-db406c55dd5d None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.311529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd68e9c-56d2-4d3f-b97b-db406c55dd5d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.313404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 215/433] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:55.314390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7affb-b2d3-4d19-b3c5-e4ea71813192 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.314670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7affb-b2d3-4d19-b3c5-e4ea71813192 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.314942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7affb-b2d3-4d19-b3c5-e4ea71813192 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.316275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 216/434] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:55.320155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7affb-b2d3-4d19-b3c5-e4ea71813192 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.403957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.415577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 219/435] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:11:55.417156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.417508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.417843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.433940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7affb-b2d3-4d19-b3c5-e4ea71813192 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.435510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:55.436797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:55.469263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.470115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.470479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.470720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.471844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.473381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.475785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 217/436] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:55.481063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.512505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 220/437] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:55.512505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.546240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.601765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7b056-33a8-4da2-bf50-9e82800d3e27 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.621353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7946d0-ea62-4743-8ef8-2f342c7d5331 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.637927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.638790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.639075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.639364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] PATH_INFO: `/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.640584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] RBAC: Authorizing `identity:get_project(project_id=d6ff204a92aa41aaa7ee6ad295eecfe9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.642033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.643756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 218/438] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/projects/d6ff204a92aa41aaa7ee6ad295eecfe9 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:55.644621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.651989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.665090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.665860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.666241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.666522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.667341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:55.667646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.669113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.675561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 221/439] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:55.676343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.787861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a3f90-19b9-4dbe-bc9e-09863f16c0f3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.810139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.824421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73ddf-a62f-40a8-875e-f8dd65312a08 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.832782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.833596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.833912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.834183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.849007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:55.858214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:55.859110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:11:55.859887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:11:55.861865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 219/440] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:11:55.868320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.873213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:55.874463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:55.874831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:55.875183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:55.879758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:55.879758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:55.881737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 222/441] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:55.887353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:55.949288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:55.984253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edffc25-fb29-434c-b7e6-491a38478ae4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:55.996064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c202d5cd-52ac-4e7c-9423-abea199de834 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.017561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.018235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.018520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.018795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.019543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:56.019847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.021317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b719d-0f95-4134-a4bb-068036077971 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.029180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 220/442] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:56.029542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.030245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.030407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17523-d09d-46e3-81b5-563ba5cdfba0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.030507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.030654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17523-d09d-46e3-81b5-563ba5cdfba0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.030762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.031137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17523-d09d-46e3-81b5-563ba5cdfba0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.031567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=8dbe145322294429a2b040dd40e9a8e0, scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:56.031878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.032308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 221/443] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:56.033358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fc64a-bfd5-4193-9da9-be53238d7231 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.034602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17523-d09d-46e3-81b5-563ba5cdfba0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.040047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 223/444] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:55 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=d6ff204a92aa41aaa7ee6ad295eecfe9&role.id=8dbe145322294429a2b040dd40e9a8e0 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:56.041256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.041546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.041825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.060801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:56.062268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:56.083038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.152116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 224/445] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:11:56.163033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17523-d09d-46e3-81b5-563ba5cdfba0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.164936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.202894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.203830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.204343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.204767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.224951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.288588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.289818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7fcc6-2169-419c-8e7b-23da71dc693d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.290112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.336326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.337062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.337343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.337618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.341153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.349150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:11:56.358529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:11:56.359426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:11:56.360242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 225/446] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:11:56.365733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.415945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 222/447] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:11:56.421880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.501437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002082cf-2c66-4431-a300-27a9513b2085 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.511141 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.533997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8ce457-37de-42a3-b59a-1c99a5b24cbe None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.542209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.543961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.543961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.543961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.544709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:56.545379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.547111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.558204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 226/448] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:56.564412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.577930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.578881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.579168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.579442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] PATH_INFO: `/v3/projects/ddf9a48b00224ac59fc18cab7ad35465` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.580728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] RBAC: Authorizing `identity:get_project(project_id=ddf9a48b00224ac59fc18cab7ad35465)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.582170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.585378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 223/449] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/projects/ddf9a48b00224ac59fc18cab7ad35465 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:56.590935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.602842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.679104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3abc4-76d9-4d18-a450-bcec6bc4daa2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.716771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a10d12-9a73-4351-b863-d3d3b36223a8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.721739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.722396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.722685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.722967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.751751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.752715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.753002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.753279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] PATH_INFO: `/v3/groups/c643507620ce49438c8f34ed271f9f16` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.754452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] RBAC: Authorizing `identity:get_group(group_id=c643507620ce49438c8f34ed271f9f16)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.756035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.757783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 224/450] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/groups/c643507620ce49438c8f34ed271f9f16 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:56.763427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.767678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.797759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.799224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.839460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:56.862397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79102b9f-a4eb-4591-8d3d-76efcaeb4b0d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:56.898548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:56.899201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:56.899485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:56.899762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:56.900625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] RBAC: Adding query filter params (group.id=c643507620ce49438c8f34ed271f9f16, role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=ddf9a48b00224ac59fc18cab7ad35465) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:56.900938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:56.901854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 227/451] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:11:56.902525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:56.907219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:56.924112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 225/452] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/role_assignments?group.id=c643507620ce49438c8f34ed271f9f16&scope.project.id=ddf9a48b00224ac59fc18cab7ad35465&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:56.980661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:57.013520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44704601-1a86-48b2-bedb-89ce79b915fa None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:57.050170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:57.051041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:57.051326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:57.051602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] PATH_INFO: `/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:57.053672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] RBAC: Authorizing `identity:get_project(project_id=98ca6d3d93034fc49fb3c9265ccf0dcb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:57.055176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:57.056915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 228/453] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:11:56 2022] GET /identity/v3/projects/98ca6d3d93034fc49fb3c9265ccf0dcb => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:11:57.062603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:57.175596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bb5778-a9a3-4eaa-8d6f-964cceb961c9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:57.209500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c7330-fee1-4ec0-8395-ecdfef06c887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:57.210354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c7330-fee1-4ec0-8395-ecdfef06c887 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:57.210628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c7330-fee1-4ec0-8395-ecdfef06c887 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:57.210897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c7330-fee1-4ec0-8395-ecdfef06c887 None admin] PATH_INFO: `/v3/groups/8712bca585aa41f9803d8411a4027b50` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:57.212280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c7330-fee1-4ec0-8395-ecdfef06c887 None admin] RBAC: Authorizing `identity:get_group(group_id=8712bca585aa41f9803d8411a4027b50)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:57.213765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c7330-fee1-4ec0-8395-ecdfef06c887 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:57.215391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 226/454] 10.222.0.17 () {56 vars in 1180 bytes} [Fri Jul 15 19:11:57 2022] GET /identity/v3/groups/8712bca585aa41f9803d8411a4027b50 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:57.221266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:57.321484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34f4aa-074c-4c6b-b6f2-67c7145ac6eb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:11:57.354060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:11:57.354712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:57.354986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:57.355259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:57.356073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] RBAC: Adding query filter params (group.id=8712bca585aa41f9803d8411a4027b50, role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=98ca6d3d93034fc49fb3c9265ccf0dcb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:11:57.356409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:11:57.357847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8c9abd-58be-4584-8656-7e14bbb55459 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:11:57.363404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 229/455] 10.222.0.17 () {56 vars in 1376 bytes} [Fri Jul 15 19:11:57 2022] GET /identity/v3/role_assignments?group.id=8712bca585aa41f9803d8411a4027b50&scope.project.id=98ca6d3d93034fc49fb3c9265ccf0dcb&role.id=56597e05853f4660b09a96c651dd0c7a => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:11:59.762201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb20bd-07da-47b1-9cf2-99fb3b1cdcb9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:59.763131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb20bd-07da-47b1-9cf2-99fb3b1cdcb9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:59.763593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb20bd-07da-47b1-9cf2-99fb3b1cdcb9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:59.765435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 227/456] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:11:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:11:59.776368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4882e-6ccf-459b-bcd1-e51fa65b789e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:11:59.776806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4882e-6ccf-459b-bcd1-e51fa65b789e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:11:59.776970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4882e-6ccf-459b-bcd1-e51fa65b789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:11:59.787942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:11:59.805758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-20f4882e-6ccf-459b-bcd1-e51fa65b789e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:11:59.806963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-20f4882e-6ccf-459b-bcd1-e51fa65b789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:11:59.905713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 230/457] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:11:59.957844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb20bd-07da-47b1-9cf2-99fb3b1cdcb9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:11:59.972836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:00.092898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb20bd-07da-47b1-9cf2-99fb3b1cdcb9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:00.141718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:00.142515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.142797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.143071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.158609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:00.160020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6decb2-9022-43a3-9b16-ccadda5f2895 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.160198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:00.160700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6decb2-9022-43a3-9b16-ccadda5f2895 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.161169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6decb2-9022-43a3-9b16-ccadda5f2895 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.163478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 231/458] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:00.169024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db9910-7d40-4593-8435-d251c49501de None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.169560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db9910-7d40-4593-8435-d251c49501de None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.170073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db9910-7d40-4593-8435-d251c49501de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:00.182553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:00.183468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:00.184917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:00.184917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:00.184917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:00.184917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:00.184917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 228/459] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:00.193965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.201476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-84db9910-7d40-4593-8435-d251c49501de None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:00.203712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-84db9910-7d40-4593-8435-d251c49501de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:00.277591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:00.308182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 232/460] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:00.309579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42586-d5af-420d-a360-294e69f32a99 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.309873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42586-d5af-420d-a360-294e69f32a99 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.310156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a42586-d5af-420d-a360-294e69f32a99 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.311715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 233/461] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:00.321801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.322085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.322359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.342552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3b7c1-43cb-4c86-82ef-de4ebe4a485b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:00.345295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:00.346639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:00.378230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:00.379145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.379566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.379975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.380880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:00.381325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:00.382849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:00.401833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 229/462] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:00.402880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.414469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:00.432211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 234/463] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:00.433233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.509585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b448062-dd08-4f6e-b898-1ae25ae286d4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:00.531013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:00.548513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:00.549800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.550231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.560633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b1906-a3aa-4deb-ba42-00130c7999b6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:00.565870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:00.567340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:00.595101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:00.596058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:00.596940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:00.596940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:00.596940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:00.596940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:00.596940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 230/464] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:00.597629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.601121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124cc152-8ca7-4677-8f9e-4a5688c61637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:00.602273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cc152-8ca7-4677-8f9e-4a5688c61637 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.602569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cc152-8ca7-4677-8f9e-4a5688c61637 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.602855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124cc152-8ca7-4677-8f9e-4a5688c61637 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.603573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cc152-8ca7-4677-8f9e-4a5688c61637 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:00.605121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124cc152-8ca7-4677-8f9e-4a5688c61637 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:00.636289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 235/465] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:12:00 2022] POST /identity/v3/users => generated 246 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:00.637838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c12b5c-f96c-4fdb-b04b-f1701f9c606a None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.638132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c12b5c-f96c-4fdb-b04b-f1701f9c606a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.638581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c12b5c-f96c-4fdb-b04b-f1701f9c606a None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.644866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 236/466] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:00.649157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c12b5c-f96c-4fdb-b04b-f1701f9c606a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.720384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:00.733826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baada3-0e4c-4165-ad37-e0401cd94f81 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:00.778292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:00.779682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.780165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.780654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.781514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c12b5c-f96c-4fdb-b04b-f1701f9c606a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:00.793446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:00.795256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-dd752dbd-5ccc-4b20-b492-a6cc7c5e0e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:00.814570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:00.815523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:00.816412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:00.816412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:00.816412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:00.816412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:00.816879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 231/467] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:00.818853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.819712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.819712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.826645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:00.827608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:00.828119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:00.828601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:00.830649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:00.831160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:00.832823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:00.845293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:00.846138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:00.850541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 237/468] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:00.850806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.870208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:00.935308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 232/469] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:00.936556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:00.998122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500a608b-0705-4a5f-b1a8-fe17d7f65902 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.008990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:01.037996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.039023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.039522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.039969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.040874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:01.041363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.042934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.055864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 238/470] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:01.057123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:01.067179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8c7bb5-7895-4fce-bf8e-45ce46bd002d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.106834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.107814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.108327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.108795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.109688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.111450 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.134172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 233/471] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:12:00 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:01.135549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:01.172689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:01.198222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d69f2e-5112-41f9-88f3-c699b2e6138d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.249412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.250909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.251645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.252174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.269184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.272313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.295596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff7a80-6524-48fb-9a1c-da4b71a22e58 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-2f3d4af4-55e3-421c-b6a2-bdb0e2e0d553 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:01.299085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:01.300168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:01.301227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:01.301227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:01.301227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:01.301227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:01.304151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 239/472] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:01.305633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15ac1a-c942-4917-90b7-dc9572f7470b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.306687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15ac1a-c942-4917-90b7-dc9572f7470b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.307108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15ac1a-c942-4917-90b7-dc9572f7470b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.309321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 240/473] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:01.310187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15ac1a-c942-4917-90b7-dc9572f7470b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:01.339447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:01.350247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd9572-0056-4b41-9569-a90d8322599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.351140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd9572-0056-4b41-9569-a90d8322599e None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.351245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd9572-0056-4b41-9569-a90d8322599e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.352117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd9572-0056-4b41-9569-a90d8322599e None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.352241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd9572-0056-4b41-9569-a90d8322599e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.354049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd9572-0056-4b41-9569-a90d8322599e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.383661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 234/474] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:12:01 2022] POST /identity/v3/users => generated 248 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:01.385641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.386144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.386923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.412959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:01.416666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:01.461656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15ac1a-c942-4917-90b7-dc9572f7470b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.493722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:01.508264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 235/475] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:01.511776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.512725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.513209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.513694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.514637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:01.515150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.516780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.528400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:01.530882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 241/476] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:01.535867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:01.644962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:01.675192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f352a-07ea-4e40-b200-17acf0d630a1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.693959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2e221-96d7-4d1a-8290-7241078da641 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.726391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.727660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.728448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.729323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.730499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.733071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.736584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.737250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.737530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.737804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.738550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:01.740481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:01.752574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.sql.core [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 19:12:01.752574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 19:12:01.752574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [parameters: {'id': '78ba99e9fe1844f1b9232f2241a67878', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 19:12:01.752574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80200) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 19:12:01.768134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 242/477] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:12:01 2022] POST /identity/v3/users => generated 249 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result.read() Jul 15 19:12:01.773954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return ref.to_dict() Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:12:01.774840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application yield resource Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise self.value Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application session.commit() Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 19:12:01.775767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.session.flush() Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application compat.raise_( Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 19:12:01.776770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:12:01.777636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result.read() Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:12:01.778885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [parameters: {'id': '78ba99e9fe1844f1b9232f2241a67878', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:01.779676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 15 19:12:01.782246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:01.784493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 236/478] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Jul 15 19:12:01 2022] POST /identity/v3/roles => generated 144 bytes in 248 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:01.786317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:01.907348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84c95c-7305-4ae6-a6d9-968fb5025bd3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:01.945396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912b1f99-69c0-442b-ba8e-2b40218f028b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:01.946221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b1f99-69c0-442b-ba8e-2b40218f028b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:01.946507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b1f99-69c0-442b-ba8e-2b40218f028b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:01.946784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912b1f99-69c0-442b-ba8e-2b40218f028b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:01.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:01.959227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912b1f99-69c0-442b-ba8e-2b40218f028b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-912b1f99-69c0-442b-ba8e-2b40218f028b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:01.967534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:01.968471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 15 19:12:01.969274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:01.973791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 243/479] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:01.975878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:02.050543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:02.100273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d1bf1-6ca6-4cdb-8455-a4dd6148d521 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:02.138185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:02.138860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:02.139139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:02.139414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:02.140197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:02.140503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:02.141950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e3175-bf95-4a83-bcb6-e72d34439cc4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:02.153978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 237/480] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:03.058546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89a5a-8d95-4af9-94fc-606545cf6e65 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.059030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89a5a-8d95-4af9-94fc-606545cf6e65 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.059030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f89a5a-8d95-4af9-94fc-606545cf6e65 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.060635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 244/481] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:03.066220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.066616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.066786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.097796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:03.098970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:03.198658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 238/482] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:12:03.233155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f89a5a-8d95-4af9-94fc-606545cf6e65 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:03.384126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f89a5a-8d95-4af9-94fc-606545cf6e65 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:03.432846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:03.433676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.433974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.434252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.449062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:03.456908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:03.457799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:03.459310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 245/483] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:03.465174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:03.555267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:03.620002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e89939-fab6-4970-a207-8712730cba70 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:03.664004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:03.664617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.664892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.665163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.665907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:03.666207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:03.667575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a8e73-8564-4a92-a720-12be0efdd417 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:03.683352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 239/484] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:03.688914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:03.768509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:03.819612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6fd4c-a7bf-4691-91f4-0841f82689c4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:03.819798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3c5a-d9e9-41c6-9b80-51e28dc67027 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.819798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3c5a-d9e9-41c6-9b80-51e28dc67027 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.819798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c3c5a-d9e9-41c6-9b80-51e28dc67027 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.820466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 240/485] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:03.825639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d73c20-67ad-4c7f-9002-40f198f3699b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.826219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d73c20-67ad-4c7f-9002-40f198f3699b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.826789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d73c20-67ad-4c7f-9002-40f198f3699b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.844338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-11d73c20-67ad-4c7f-9002-40f198f3699b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:03.845875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-11d73c20-67ad-4c7f-9002-40f198f3699b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:03.860265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:03.860941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.861221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.861496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.882307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:03.882307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:03.910350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:03.911258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:03.912858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:03.912858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:03.912858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:03.912858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:03.922184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 246/486] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:03.925097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:03.970287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 241/487] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:12:03.971958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e9a1-c300-4ef9-8104-01738c2e2c58 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.976527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e9a1-c300-4ef9-8104-01738c2e2c58 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.977025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e9a1-c300-4ef9-8104-01738c2e2c58 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:03.978796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 242/488] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:03.997070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:03.997657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:03.998158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.021444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:04.022965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:04.049844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74bd012-b17f-411d-b951-07a2aedd9284 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.059123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.064472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.088895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.089789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.090279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.090724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.091641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:04.092184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.093748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.104218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 243/489] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:12:04.107017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.114405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 247/490] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:04.124826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.237910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013bb31-9875-4c14-b701-e904f072d63d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.257661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b762a-c978-4f4c-a6cc-bd377bea8da3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.277912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.279046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.279524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.279998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.291127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:04.299286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:04.300196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:04.301008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:04.301622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.301707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.302266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.302515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.304225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 244/491] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:04.305280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.316171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.318598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:04.337384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:04.338313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:04.339222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:04.346189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 248/492] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:04.347184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.347769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.379581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.425558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3685f-f388-4993-9f72-b9325584842b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.462194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.463383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.463737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.468195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.482332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402dda5-74d4-4cc8-9ebc-3a8b2feb016c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.483125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:04.491854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:04.492891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:04.493883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:04.501483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 245/493] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:04.502664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.529439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.530710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.531084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.531436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.532450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:04.532969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.534808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.552532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 249/494] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:04.552532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.625148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.632919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d319f-06d3-430f-bffa-f8b850ce6f55 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.663104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.670267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.670936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.671225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.671493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.672364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:04.672675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.674129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.674805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562697c-dd7d-41e8-8e32-7b8ea8ed36f9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.712102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 246/495] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:04.712950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.718855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.720973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.721446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.721920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.722862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:04.723343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.724995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492d37c0-da79-493b-adc0-4983ffe03dde None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.742313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 250/496] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:04.743981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c94b3-912a-4289-bc6b-a21a12247848 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.744521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c94b3-912a-4289-bc6b-a21a12247848 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.745029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c94b3-912a-4289-bc6b-a21a12247848 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.746482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 251/497] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:04.747058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c94b3-912a-4289-bc6b-a21a12247848 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.827314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d70d7-645c-441c-ba64-6f2978c6d9fe None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.854043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c94b3-912a-4289-bc6b-a21a12247848 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:04.859937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.860752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.861029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.861295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.871716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.873204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.888571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1dc0e-58c0-4e78-b4ba-6e012308a1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:04.889218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1dc0e-58c0-4e78-b4ba-6e012308a1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:04.889516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1dc0e-58c0-4e78-b4ba-6e012308a1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:04.889778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1dc0e-58c0-4e78-b4ba-6e012308a1a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:04.891430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:04.892302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:04.893132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:04.893132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:04.893132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:04.893132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:04.893132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 247/498] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:04.893673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:04.897147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.913877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:04.960499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1dc0e-58c0-4e78-b4ba-6e012308a1a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:04.961955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd1dc0e-58c0-4e78-b4ba-6e012308a1a6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:04.993680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2565e57-22cc-4a1d-a7c9-cbf3b014c85a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.026760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.027545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.027832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.028153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.039052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.040532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-676185cf-2935-4454-949b-d19b0a9e5eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:05.058315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:05.059159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:05.059997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:05.059997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:05.059997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:05.059997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:05.059997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 248/499] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:05.060300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 252/500] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:05.061519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc1f9-c3bd-4585-a80e-b095a5465b82 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.061802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc1f9-c3bd-4585-a80e-b095a5465b82 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.062079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bc1f9-c3bd-4585-a80e-b095a5465b82 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.066466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 253/501] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:05.066466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc1f9-c3bd-4585-a80e-b095a5465b82 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.066643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.066643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.066643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.082068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:05.083257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:05.118189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.134863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.152198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 249/502] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:05.153025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.167202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bc1f9-c3bd-4585-a80e-b095a5465b82 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.201171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.201834 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.202118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.202393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.203112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:05.203417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.204828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.216198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 254/503] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:05.216198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.254729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8dfa-7679-40d5-ab2c-5c9ef6c190b7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.289530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.290313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.290591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.290870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.300592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.302044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.319980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385dc60b-63c9-4947-b765-cd3165866357 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-48248090-294e-49ba-b15c-76816c248b4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:05.320139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:05.320968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:05.321798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:05.321798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:05.321798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:05.321798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:05.321798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 250/504] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:05.322655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.322944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.323225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.334584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.340059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:05.341286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:05.356281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.356939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.357218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.357486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.358198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:05.358496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.359867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.371389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 255/505] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:05.372188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.372422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.410819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 251/506] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:05.411659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.476993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea362d6-6190-448a-aa16-eb6eaf272cd9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.511651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.512403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.512685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.512958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.513148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03058534-c850-4c8c-9e83-3979aa6b3af1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.523552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:05.531256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:05.532347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 256/507] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:05.533362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.551888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.552751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.553020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.553297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.560383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.561840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.568064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:05.579067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:05.580193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 252/508] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:05.581429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.616655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.644007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01236eb-8f4e-4f89-83f1-d256d45864b2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.677521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.678187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.678473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.678751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.679479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:05.679788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.681218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.688443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fec604-fa12-47f5-8152-7772e668701e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.692734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 257/509] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:05.693896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.725492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.726276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.726559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.727066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.733778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.735240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:05.749954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:05.750954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 253/510] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:05.757021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.789298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.799648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f25d1d-0b60-44c9-b2b0-ebab47babe8a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.833077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.834621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.834898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.835171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.840649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:05.845355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.846840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.854320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccb106-a48a-4ac8-a872-e02e10578dd6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:05.864104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:05.864955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:05.865766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:05.865766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:05.865766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:05.865766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:05.865766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 258/511] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:05.866254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.891793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:05.892591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:05.892947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:05.893298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:05.894124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:05.894522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:05.896527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:05.904897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 254/512] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:05.905608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:05.967930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ebfc5-5022-4d11-a736-8eee294c4ba6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.001637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.002293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.002573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.002846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.003267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4774b-3b78-4079-b5d1-0a22288f26cc None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.003671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:06.003974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.006129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.036993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.037719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.038001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.038273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.042965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 259/513] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:06.047555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.047631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.047631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.049250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:06.062134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:06.063148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 255/514] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:06.064204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.076428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.154061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf10d8-af84-4619-80c8-b3b6268349de None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.165858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e15fca-fdb7-462d-b9ff-1d74b41ea46c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.187148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.187815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.188132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.188416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.189251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:06.189581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.191033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.198618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.199265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.199545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.199815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.200558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:06.201139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.202497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.207824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 260/515] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:06.209479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.222451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 256/516] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:06.222451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.265629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.286146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.307065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830cd0c5-f160-43eb-9a92-d6d3770cb146 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.316858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2de236e-d387-4706-bde1-b1c4c9a86875 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.340248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.341041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.341328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.341603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.351158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.351797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.352095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.352382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.352480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.353864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:06.375198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:06.376050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:06.376934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:06.376934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:06.376934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:06.376934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:06.376934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 261/517] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:06.377190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.425254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.426827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.475126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c2d6ce-f66e-4560-bc94-33a0b375119f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.487626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.494421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.510002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.510657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.510935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.511211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.511992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:06.512326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.513753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.526889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 257/518] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:06.527707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.535033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 262/519] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:06.536137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.625856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ac7c70-ba75-43f2-8310-bf71138da507 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.642011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75563eb3-3439-4243-a1ed-2074ecf0d171 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.659706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.660383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.660664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.660936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.676358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.677652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.677933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.678386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.679081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.680563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.700946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.702356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 263/520] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:12:06 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:06.703167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.734749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.735311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.737890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.836251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d02af-7285-40aa-a577-bb5153888804 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.864099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 258/521] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:06.865396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.876744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:06.877841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:06.877841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:06.877841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:06.879288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:06.879576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:06.881966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:06.895394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 264/522] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:06.896401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:06.944263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:06.974061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f28e0d-ea8d-4121-86ab-4b689bb519c3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:06.993155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.007816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121456f-f3d9-4a8f-a29f-19cfe8648999 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.009552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.010350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.010631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.010904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.022163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.023617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:07.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:07.042728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:07.043560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:07.043560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:07.043560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:07.043560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:07.043560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 259/523] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:07.043854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.048268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.048938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.049221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.049494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.050332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=37f8217df17a44a989ce1c7ab8a9ac1a) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:07.050637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.052180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.068135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 265/524] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:06 2022] GET /identity/v3/role_assignments?user.id=37f8217df17a44a989ce1c7ab8a9ac1a&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:07.068135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.178304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666fb3ea-d142-4e13-9b3b-5a59ac7c7c1a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.209374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.216503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaafc5-1e8e-45c9-8e6b-f5fd5badd7b5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.227059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.227968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.228304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.228593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.239293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.241461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:07.261716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:07.262622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:07.263465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:07.263465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:07.263465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:07.263465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:07.264695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.265654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.266014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.266192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 260/525] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:07.266295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.267245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.275897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.277739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:07.300066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:07.301016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:07.301991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 266/526] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:07.302551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.318481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.382401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7675a8-32fe-494d-b6b4-8de7c03bdf56 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.408903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb75923-589d-415c-8330-1ffe8e97a980 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.421393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.422474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.422955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.423424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.424497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:07.425024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.426695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.440175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 261/527] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:07.441538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.447153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.447869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.448200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.448530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.449272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:07.449580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.451060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.464403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 267/528] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:07.465150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.481584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.540547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.559241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da532a3-ea2b-42d9-83ee-15a6d2f46fb8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.572722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b545ed9-9a35-4031-b711-6a4b160018b5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.600394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.601541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.602020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.602584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.603720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:07.604310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.606011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.608815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.609844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.609953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.610173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.618308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.619778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.627372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 262/529] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:07.628447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:07.636250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:07.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:07.637924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 268/530] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:07.638828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.731846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.743116 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d5320-a2ec-433f-bfb0-b3bb69e12bad None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.749136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbe113-f403-4ef0-9846-2aae3abfb34a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:07.789899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:07.792426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.793431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.793719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.793999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.798147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:07.799023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:07.799552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:07.800017 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:07.801724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.803886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:07.822463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:07.823372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:07.824239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 263/531] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:07.824692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:07.890657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:07.893025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:07.962582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a061d0-51dd-46f9-ae5e-11f73d2da709 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.004333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.005064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.005373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.005714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.006508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:08.006813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.008382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.013715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 269/532] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:08.014609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.027355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 264/533] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:08.032387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.046527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:08.050120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:08.122149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe3ecb0-1540-419b-93a4-84a1d9259acf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.144266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f73db4-bd0b-452a-affb-b08f7c7acb5c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.160542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.161231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.162251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.162349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.163076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:08.163382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.165244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.183245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 270/534] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:08.183245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.188751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba7859-291f-4775-8710-128236f837ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.190147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba7859-291f-4775-8710-128236f837ca None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.190672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba7859-291f-4775-8710-128236f837ca None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.191166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba7859-291f-4775-8710-128236f837ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.203208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba7859-291f-4775-8710-128236f837ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.205256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba7859-291f-4775-8710-128236f837ca None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-94ba7859-291f-4775-8710-128236f837ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:08.226396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:08.227314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:08.228181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:08.228181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:08.228181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:08.228181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:08.230512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 265/535] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:08.231508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba7859-291f-4775-8710-128236f837ca None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.282496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:08.292795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f49180-f396-44ff-89f6-6fcfaecbb784 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.303636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:08.334938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.335603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.335891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.336198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.337147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=6802452aab2f43d4b965be689758b611) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:08.337478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.339042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.350459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 271/536] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/role_assignments?user.id=6802452aab2f43d4b965be689758b611&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:08.350459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.353201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba7859-291f-4775-8710-128236f837ca None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.416670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.417763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.418337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.418810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.420014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=43fda9627eb84093bb59ceb1a553baf7) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:08.420579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.422329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.433697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 266/537] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/role_assignments?user.id=43fda9627eb84093bb59ceb1a553baf7&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:08.494597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00d03e-1091-45c9-b558-66a249ee068e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.555023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.555635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.555923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.556237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.556989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:08.557300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.558744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.576763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 272/538] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:08.584715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.698096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d4b52-a458-43e6-ae9e-780e847c0c20 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.732446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:08.737647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.738453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.738814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.739043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.750242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.751728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b1628c37-e51f-4706-9a37-2defef788c5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:08.767484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:08.768395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:08.769385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 267/539] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:08.779643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:08.806845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:08.847244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40551250-0bec-4e65-9658-7314640a595e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.847578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40551250-0bec-4e65-9658-7314640a595e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.847807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40551250-0bec-4e65-9658-7314640a595e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.849572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 268/540] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:08.869887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.870200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.870445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.897449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:08.898566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:08.908965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c26cf8-8a1a-41f3-958c-9073617872a4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:08.945127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:08.946158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:08.946644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:08.947129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:08.948133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:08.948634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:08.950264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:08.963304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 273/541] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:08.968974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.056486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 269/542] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 15 19:12:09.072252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:09.091114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.113099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6dd7d-8b1f-4afa-988b-4f6ea854fb65 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:09.183105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:09.183761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.184041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.184347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.185178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=9f135ce75fbc4b12977cc8f009083ff9) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:09.185483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:09.186920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1acae7d-1d33-4f8d-b76d-0ccc8bf6ba1e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:09.199731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 274/543] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:08 2022] GET /identity/v3/role_assignments?user.id=9f135ce75fbc4b12977cc8f009083ff9&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:09.281673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18041658-f27d-4eb7-8bb3-6503dd1ec815 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:09.301520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b75eb-b410-49a5-ad99-1162788ea2f5 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.302073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b75eb-b410-49a5-ad99-1162788ea2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.302145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b75eb-b410-49a5-ad99-1162788ea2f5 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.303522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 275/544] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:09.308772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:09.313655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.313933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.314218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.330871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:09.331607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.331901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.332209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.347711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:09.349504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:09.355639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:09.363666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:09.366495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:09.368438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 270/545] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:09.371713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.452888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 276/546] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:09.476802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.504553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de8351-2a99-4661-885e-b1f9d74d68b3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:09.549529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:09.550404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:09.552538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.552939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.553312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.554165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:09.554573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:09.556057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:09.566716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 271/547] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:09.577046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.610699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f606cf-83eb-4e02-a884-e42a5f1af04f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:09.664695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:09.684553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:09.685368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.685654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.685934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.705742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:09.713969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:09.714863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:09.715615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 277/548] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:09.725166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.779712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdbeb2c-8ba2-4ea4-bdb8-89fed489f325 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:09.854453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32065d0-ae84-4b5c-abf3-387ecccd4ec2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:09.867063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:09.867892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.868200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.868476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.887276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:09.888430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:09.890055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:09.905336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:09.906245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:09.906700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:09.907112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:09.908077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:09.908538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:09.910122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:09.924826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:09.926352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:09.928106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:09.928106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:09.928106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:09.928106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:09.931307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 278/549] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:09.932558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 272/550] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:09.936747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:09.938527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.109086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:10.123227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd04f40-5618-4c35-91ce-8e7d5597013e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.128904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0a65d-ffa0-429e-b370-4d3403574e04 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.194992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.195750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.196039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.196343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.197082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:10.197393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:10.198774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:10.201766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.202571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.202853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.203128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.220104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 273/551] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:10.222605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.228952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:10.230451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:10.265466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:10.266370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:10.267366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:10.267366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:10.267366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:10.267366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:10.272144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 279/552] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:10.275485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.352197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:10.423360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddb6ee-db0d-4f75-bded-528526e28cb4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.459563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50da2ba-316f-4517-a020-b1df28261032 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.482983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.483772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.484050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.484350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.491674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:10.493162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:10.508550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:10.509508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:10.510441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 274/553] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:10.515588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.530605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.531273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.531553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.531824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.533090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:10.533403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:10.534789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:10.548281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 280/554] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:10.554690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.555385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:10.666022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0f54f-fd25-426b-9d43-6f3ba6f78e6b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.703350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.703940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.704254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.704537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.705379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:10.705689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:10.707332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:10.725236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 275/555] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Jul 15 19:12:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:10.732557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.789611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fbb65-a476-449c-b4f0-a67cda20bf60 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.794113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:10.850412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.851217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.851504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.851785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.865728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:10.867216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:10.880880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-0003a8ec-0fd7-44a9-9c04-e744e491c4a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:10.898779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:10.899859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:10.901238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 281/556] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:12:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:10.902314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29bad3-6167-4f67-aa27-bb7b47c0856d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.904953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29bad3-6167-4f67-aa27-bb7b47c0856d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.904953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29bad3-6167-4f67-aa27-bb7b47c0856d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:10.905266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 282/557] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:10.910670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29bad3-6167-4f67-aa27-bb7b47c0856d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:10.919025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4557d98-7e1c-4923-9096-b17a1ad3253a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:10.957096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b224f0b5-829d-49ea-ae4a-159fe4a5c447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:10.958110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224f0b5-829d-49ea-ae4a-159fe4a5c447 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:10.958401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224f0b5-829d-49ea-ae4a-159fe4a5c447 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:10.958672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b224f0b5-829d-49ea-ae4a-159fe4a5c447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.022101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29bad3-6167-4f67-aa27-bb7b47c0856d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.031021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224f0b5-829d-49ea-ae4a-159fe4a5c447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.032520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b224f0b5-829d-49ea-ae4a-159fe4a5c447 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.058928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.059595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.059876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.060186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.061025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:11.061328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.062765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d7283-62bf-47f4-a952-3c5b217cff8a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.081006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 283/558] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:12:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:11.082548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.082780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.083098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.101622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:11.102815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:11.108344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.123073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.147776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 276/559] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:11.149139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558ede-ee65-464c-8a13-1502506d4ec9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.149432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558ede-ee65-464c-8a13-1502506d4ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.150029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02558ede-ee65-464c-8a13-1502506d4ec9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.155447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 277/560] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:11.155953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558ede-ee65-464c-8a13-1502506d4ec9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.179791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 284/561] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:11.180579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.278382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02558ede-ee65-464c-8a13-1502506d4ec9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.287087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ffbb1-579b-4f9d-8c31-268f469406fb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.317744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.319570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.320328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.320808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.324515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.325319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.325606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.325881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.331702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.334405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.340841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.342715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-0aee4102-292d-45e9-9142-079b12627da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:11.352934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:11.353986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:11.354998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:11.354998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:11.354998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:11.354998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:11.354998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 278/562] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:11.355276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.355543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.355817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:11.369287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:11.370567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:11.372420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:11.372420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:11.372420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:11.372420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:11.372420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 285/563] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:11.372420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.375488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:11.377236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.378597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:11.398290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.454650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 279/564] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:11.456291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.488915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd273f8-4099-45a7-9cbe-9ced1321def6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.527815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.528703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.528922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.529189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.539963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:11.547750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:11.548920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 286/565] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:11.549981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.573845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1561100b-7e79-49a5-8831-8182ceb4779d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.608269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.608931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.609223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.609489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.610213 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:11.610521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.611338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.611922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.624624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 280/566] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:11.625261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.639338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.654412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342cad0e-63a2-4b98-b360-c5de1f469435 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.688554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.689214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.689492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.689764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.690478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:11.690782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.692172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f1a906-905c-4bce-b149-c0af4fdc0979 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.704703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 287/567] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:11.705956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8464b-8084-4990-9d40-131173671805 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.706244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8464b-8084-4990-9d40-131173671805 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.706527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8464b-8084-4990-9d40-131173671805 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.707756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 288/568] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:11.709105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8464b-8084-4990-9d40-131173671805 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.730420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703409b-dcd1-47ae-92fc-af445685b6a6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.765852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.766711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.766997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.767325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.779286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:11.787647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:11.788780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 281/569] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:11.789728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.814581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8464b-8084-4990-9d40-131173671805 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.838559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.850331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.851000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.851288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.851568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.852355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:11.852674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.854054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.860603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:11.862054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 289/570] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:11.862835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:11.893345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d6178-e97f-4d9b-bcb5-024d05bedbad None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.927066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:11.928229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.928523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.928804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.935511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:11.936997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-09a04dd6-4ff2-4932-aa40-90eddd30df9e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:11.951845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:11.952946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:11.953821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 282/571] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:11.954496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:11.954790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:11.955042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:11.967924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1be29-edc8-4536-91ac-45288d7173bc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:11.971352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:11.972568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:12.003417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.004247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.004532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.004806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.012166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.013626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:12.029129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:12.030189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:12.031450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 290/572] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:12.032887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.040819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 283/573] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:12.041648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.088727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.104261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.143311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38716dda-d103-46df-aaad-2bed4e634e69 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.143495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e5090-a7ff-48a6-8d3b-5fefb7b8e6a7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.177637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.177807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.178312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.178406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.178595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.178684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.178868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.178956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.179654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:12.179919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.181316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.188806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.189489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 291/574] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:12.190171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.190252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:12.214888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:12.215977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:12.217091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:12.217091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:12.217091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:12.217091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:12.217091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 284/575] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:12.217424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.297931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11307ce6-6305-4481-adff-326b9eead5e0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.306710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.328362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba442d5-768e-488b-aacb-271e8e0c4b31 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.331537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.332230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.332515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.332791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.334413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:12.334726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.336181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.342652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.348396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 292/576] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:12.349850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.364174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.365085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.365322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.365601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.366389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:12.366698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.368255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.379896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 285/577] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:12.380846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.450240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680811c5-347f-4b76-899a-adbd041ae599 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.481267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090b380-4283-42ba-ae62-da865f94ddce None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.483740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.484561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.484844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.485118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.495284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:12.503221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:12.504180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:12.505721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 293/578] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:12.506867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.516390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.517175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.517454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.517725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.528675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.530121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.538111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:12.547043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:12.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:12.549063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:12.549063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:12.549063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:12.549063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:12.549063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 286/579] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:12.549063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.576500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.610252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc2ff1-b897-45c6-b09c-93947d03a01d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.643250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.643907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.644221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.644496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.645211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:12.645556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.647081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.649537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4b094-efda-4a8d-9a18-9ffc9a81f2a3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.658947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 294/580] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:12.660166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.690300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.691188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.691473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.691752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/37f8217df17a44a989ce1c7ab8a9ac1a/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.706232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=37f8217df17a44a989ce1c7ab8a9ac1a, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.707697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.720728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 287/581] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:12 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/37f8217df17a44a989ce1c7ab8a9ac1a/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:12.721517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.757512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.771803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f9976-d282-4ae4-adfa-4b68c9f09653 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.812694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.813341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.813633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.813911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.814781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=8b3c45809bac4828bcde30494ab7a0d2) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:12.815089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.816596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.832119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 295/582] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/role_assignments?user.id=8b3c45809bac4828bcde30494ab7a0d2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:12.832119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.853197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:12.888532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac837f0-69ab-40a9-8e02-17df02a2dd74 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:12.936734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:12.937432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:12.937717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:12.937995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:12.939176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:12.939489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:12.941262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:12.953389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 288/583] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:12.954393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:12.977645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8e93c-db44-4318-93ce-ef1860aa13bb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.025124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.025953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.026245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.026526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.027587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:13.027908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.029784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.047371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 296/584] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:13.052384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.065545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.081708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e9297-0719-4847-8b4b-38dc5e906d75 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.119140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.119939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.120261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.120541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.129305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.131320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:13.146819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:13.147807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.149102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 289/585] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:13.156522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.164448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839920f-8fb1-463b-ae47-fb0574643423 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.202543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.203728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.204306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.204779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.218541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.220015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:13.239642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:13.240620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:13.241476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:13.241476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:13.241476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:13.241476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:13.244490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 297/586] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:13.245458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.285783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf779a7-7e9d-46bd-bf48-8e8684c468a2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.318015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.327463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.328552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.329691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.330162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] PATH_INFO: `/v3/users/glance` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.345653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.348171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.365669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0d4b70-055f-43dc-b92c-8061a1c4cbc4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:13.367890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:13.369007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:12:13.370133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:13.372992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 290/587] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:13.374212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.418468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.419389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.419819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.420280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.421819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:13.422280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.424343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.441522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.455109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 298/588] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:13.455109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.495164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac61f1-b995-4a0c-8a6b-d4cc602f3cfb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.531772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.532626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.533004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.533377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.534223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:13.534626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.537159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.554624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 291/589] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:13.554624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.596919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19778d-5972-4961-aa22-cb7fc0b9fb42 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.628826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.646874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.647742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.648309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.648729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.650249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:13.650838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.652853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.663028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680371de-7899-48e7-87d2-ddb281392f1f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.676065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 299/590] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:13.677680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.684205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.705737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.707223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.708589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.709018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.723102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.725467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:13.751830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:13.752955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:13.754019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:13.754019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:13.754019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:13.754019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:13.761108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 292/591] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:13.761108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.804880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc150385-dc30-47da-9eb1-aaa99bf63608 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.840798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.841751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.842193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.842540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.850234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.851763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:13.867862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:13.869066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:12:13.870175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:13.876721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 300/592] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:13.876721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:13.901038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7833fb-256b-4c51-916f-6def8f542d54 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:13.935952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:13.944392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:13.945659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:13.946207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:13.946732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:13.959903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:13.961921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:13.985491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:13.986446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:13.987309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:13.987309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:13.987309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:13.987309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:13.990793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 293/593] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:13.991956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.008259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a9db6-a434-4606-b7c2-4cd1138e39ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.018579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.053088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.053758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.054043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.054345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.055190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:14.055502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.057190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.074574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 301/594] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:14.074574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.121099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6264a4e1-211b-4091-81d2-b67ade6a2b5a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.167702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.168612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.169042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.169484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.170980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:14.171730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.174010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.180887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.184634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38b421-0dc8-4377-ae59-a7a1fc138b3e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.203283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 294/595] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Jul 15 19:12:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:14.203283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.232802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.234035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.234710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.235141 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.236230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:14.236988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.238876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.255894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 302/596] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:14.257187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.298042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.322372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a28bbb-cb6b-4144-81a3-108db7842906 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.363927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.365260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.365635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.366023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.374100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.375714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.388073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bac588-25f0-44d6-9f78-1516ca8628c2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:14.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:14.392649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:14.393592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:14.395730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 295/597] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:14.396724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.427893 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.428960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.429391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.429905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.441238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.443598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:14.462957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:14.464397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:14.465326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:14.465326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:14.465326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:14.465326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:14.467255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 303/598] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:14.468543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.484356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.513657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c12f3f-80fa-4ba6-a9d7-fbc75b89da87 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.550857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.552540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.552939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.553318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.561123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.562790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.565558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:14.580894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:14.581926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:14.583046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:14.584279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015b7ab-802c-4434-9f56-834e445349fe None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.590273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 296/599] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:14.590273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.628266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.629156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.629701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.630309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.631577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:14.632026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.633648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.647409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 304/600] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:14.648706 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.727597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7015f619-0a00-41f0-8de1-d45a3cf8c45b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.744942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.787694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e07bbb-4199-4f70-9b89-810f1213ffb6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.790885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.791261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.791645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.792030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.792960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:14.793441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.794959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.809286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 297/601] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:14.810474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.834012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.834957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.835388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.835810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.836797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:14.837355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.838949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:14.858565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 305/602] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:14.859659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:14.882195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:14.930459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f81baa-f4db-48e2-b6f3-7e689913dcb5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.964717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:14.965613 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:14.965897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:14.966165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/43fda9627eb84093bb59ceb1a553baf7/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:14.971509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8bd25-2100-4185-975d-94e0b8066ea2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:14.981811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=43fda9627eb84093bb59ceb1a553baf7, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:14.983998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:15.001412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 298/603] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:14 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/43fda9627eb84093bb59ceb1a553baf7/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:15.002908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.008308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.009144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.009424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.009702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.013008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:15.017992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:15.019734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-88c13c2e-6604-4d26-ae2d-67a3af91920c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:15.037620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:15.038782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:15.039836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 306/604] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:15.040737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b98b9-1647-4881-905c-b037e22db20e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.041025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b98b9-1647-4881-905c-b037e22db20e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.041299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b98b9-1647-4881-905c-b037e22db20e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.046533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 307/605] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:15.046533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48698d1e-955a-436d-b78f-351accad1b4e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.046533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48698d1e-955a-436d-b78f-351accad1b4e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.046533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48698d1e-955a-436d-b78f-351accad1b4e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.046884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 308/606] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:15.047336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48698d1e-955a-436d-b78f-351accad1b4e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.169457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095e9d74-6aad-409f-9af9-177e4d301240 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.183222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48698d1e-955a-436d-b78f-351accad1b4e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.183597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:15.213209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e78d16-dc15-4997-8a77-a3e3f10e9598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.214104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78d16-dc15-4997-8a77-a3e3f10e9598 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.214389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78d16-dc15-4997-8a77-a3e3f10e9598 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.214663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e78d16-dc15-4997-8a77-a3e3f10e9598 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/6802452aab2f43d4b965be689758b611/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.219970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.221033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.221520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.222016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.223498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:15.224324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:15.226448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba2086-0335-4225-8f5d-b15748797804 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:15.228924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e78d16-dc15-4997-8a77-a3e3f10e9598 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=6802452aab2f43d4b965be689758b611, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:15.230361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e78d16-dc15-4997-8a77-a3e3f10e9598 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:15.242610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 309/607] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:15.244288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.244783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.245320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.248177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 299/608] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:15 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/6802452aab2f43d4b965be689758b611/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:15.250754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.251989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.254132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.271286 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:15.272853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:15.275297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:15.276815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:15.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:15.361292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 300/609] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:15.362684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.376105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 310/610] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:15.393609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.493608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:15.505652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fb3af-f49f-4905-80bf-dbd0d07b8f1d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.510837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d900881-a535-48a4-bb41-1d846f53ec7b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.550794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.551844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.552217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.552568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/9f135ce75fbc4b12977cc8f009083ff9/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.554791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.555986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.556497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.557744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.569878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=9f135ce75fbc4b12977cc8f009083ff9, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:15.571276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:15.571448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:15.580801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:15.581809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:15.582621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:15.586065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 311/611] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:15.587295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.589829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 301/612] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:15 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/9f135ce75fbc4b12977cc8f009083ff9/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:15.592675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.638179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:15.720449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c4bc9-d4c9-412e-be8e-332ed1609d33 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.756929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b4b34a-b940-4853-9b6b-f5f983d1b6aa None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.780913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.781645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.782042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.782354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.783198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:15.783574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:15.785070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:15.795088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 302/613] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:15.797515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:15.798543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:15.799004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:15.799541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:15.800623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.806499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:15.815304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:15.825247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:15.826194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:15.826998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 312/614] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:15.832354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:15.967010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1ddab-cc1e-4a18-bf5c-a60a040cfa4a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:15.969038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:16.012579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.015430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.015867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.016384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.018123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:16.019673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.022875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f37bf6-dfa8-4ec6-9f3b-1b1b6344332e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.025770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.042001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 313/615] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:16.047517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.102981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.104076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.104521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.104959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.129662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.132011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:16.154542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:16.155470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:16.156351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:16.156351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:16.156351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:16.156351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:16.159270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 303/616] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:16.164871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.244480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c85af-68b0-4f00-a9a5-3084219cd242 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.281410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:16.292098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.294194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.295538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.296164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.322125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.323956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:16.349948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:16.350873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:16.351771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:16.351771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:16.351771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:16.351771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:16.354957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 314/617] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:16.365444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.370926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6276b5e-c4eb-4020-81f1-a74122ee93c6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.408391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:16.415148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.415867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.416170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.416483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.417219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:16.417526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.418960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.443404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 304/618] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:16.453063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.560826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a2dd76-aaf5-4f78-9e83-8434b87a2448 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.575569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:16.592115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b53ea-6529-491b-ac60-0717d2f84de0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.609505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.610480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.610933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.611370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.624831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.627338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:16.647463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:16.648618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:16.649885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:16.650516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.650516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.650516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.650516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.650769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:16.650769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.651213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.652610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 305/619] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:16.659012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.669061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 315/620] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:16.680418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.768750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:16.812202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab92c4c-3302-4260-90a2-8fa97f411661 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.858425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4ff2b-145c-482d-8baa-c59f7e176973 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:16.876168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.877030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.877315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.877594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.889173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.890655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:16.914062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:16.915053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:16.915992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 316/621] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:16.921689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:16.938324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:16.939005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:16.939287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:16.939559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:16.940449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:16.940759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:16.942279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:16.981183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 306/622] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:16.986931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.032474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.126821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467321c-5b9d-4780-af86-d1a09e50b682 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.167866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64629b-354a-473e-8ffd-4bf423f02dd4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.174941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14266660-a4cc-4004-8491-571c91848fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.175619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14266660-a4cc-4004-8491-571c91848fec None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.175901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14266660-a4cc-4004-8491-571c91848fec None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.176208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14266660-a4cc-4004-8491-571c91848fec None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.177521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14266660-a4cc-4004-8491-571c91848fec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:17.177765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14266660-a4cc-4004-8491-571c91848fec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:17.179205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14266660-a4cc-4004-8491-571c91848fec None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:17.181891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.198448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 317/623] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:17.204073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14266660-a4cc-4004-8491-571c91848fec None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.251714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.252981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.253479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.253983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.277499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:17.279286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:17.307375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14266660-a4cc-4004-8491-571c91848fec None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:17.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:17.329110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:17.330010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:17.330010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:17.330010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:17.330010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:17.336209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 307/624] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:17.340667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.348183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3303175b-7ab5-4ef3-9211-6865320e6536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.348908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303175b-7ab5-4ef3-9211-6865320e6536 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.349454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303175b-7ab5-4ef3-9211-6865320e6536 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.349454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3303175b-7ab5-4ef3-9211-6865320e6536 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.421542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303175b-7ab5-4ef3-9211-6865320e6536 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:17.422949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3303175b-7ab5-4ef3-9211-6865320e6536 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:17.427652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.494119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.512061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3dc0a3-c8cd-4478-8983-aa8fda1f380a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.530491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 318/625] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:12:17.536843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9a92-b054-44fc-aba3-b3e4189413d6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.537290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9a92-b054-44fc-aba3-b3e4189413d6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.537579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb9a92-b054-44fc-aba3-b3e4189413d6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.538909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 319/626] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:17.540050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96ce47-2a59-40d0-9efd-6065756586a0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.540359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96ce47-2a59-40d0-9efd-6065756586a0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.540638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96ce47-2a59-40d0-9efd-6065756586a0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.544202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 320/627] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:17.544202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96ce47-2a59-40d0-9efd-6065756586a0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.553121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.554071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.554503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.554925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.555819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:17.556304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:17.557826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49907cd1-dac5-4402-b7f4-b5b37cee4923 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:17.574608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 308/628] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:17.574608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d90b22-b4ce-41cb-9f5a-9f0fd8649270 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.574608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d90b22-b4ce-41cb-9f5a-9f0fd8649270 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.574608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d90b22-b4ce-41cb-9f5a-9f0fd8649270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.589641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-17d90b22-b4ce-41cb-9f5a-9f0fd8649270 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:17.590833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-17d90b22-b4ce-41cb-9f5a-9f0fd8649270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:17.650116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.659260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96ce47-2a59-40d0-9efd-6065756586a0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.664022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 309/629] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:17.665620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.666083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.666524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.682969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:17.684221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:17.702044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.703165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.703455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.703735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.715260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:17.716871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:17.735861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:17.736827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:17.737716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:17.737716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:17.737716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:17.737716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:17.737716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 321/630] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:17.737998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.768111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 310/631] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:17.768111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.773049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.850759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b875a2f8-a7dc-4128-8aca-0d9e26b0d07e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.885014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68885fbd-fe2e-404f-911f-20e481484587 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:17.891069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.891874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.892191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.892474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.900910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:17.902379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:17.917663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:17.918752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 322/632] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:17.919638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:17.926545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:17.927525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:17.927957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:17.928264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:17.933280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:17.939909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:17.948495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:17.949603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:17.950472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 311/633] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:17.951139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.017313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.034063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca0085-395d-401d-b7eb-56c89c043014 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.056398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1d5d-bcaa-4dff-bfa8-c96286331a0b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.066757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.068106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.068419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.068699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.069429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:18.069738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.071143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.085371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 323/634] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:18.086158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.090338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.091252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.091609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.091961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.102340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:18.110032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:18.111111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 312/635] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:18.111939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.154837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.189586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ab719-a2ae-4f3f-9d97-9cff18193043 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.211768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c100eff5-e569-417b-ba9a-42eb7d5676e2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.224864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.225549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.225828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.226104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.226886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:18.227190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.228662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96768755-266d-44fd-8fc0-171466bff2b9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.235244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.240282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 324/636] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:18.241529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a2ae-a265-49f1-9946-43cb45dd19a9 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.241815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a2ae-a265-49f1-9946-43cb45dd19a9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.242091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9a2ae-a265-49f1-9946-43cb45dd19a9 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.243458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 325/637] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:18.243980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a2ae-a265-49f1-9946-43cb45dd19a9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.251302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.251972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.252295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.252575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.253326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:18.253632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.255020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.263644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 313/638] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:18.264470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.344591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9a2ae-a265-49f1-9946-43cb45dd19a9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.362775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e2f50-e993-4911-afbd-f8f0e962b9c9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.369421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.378512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.379296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.379599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.379885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.386620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.388073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.396744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.397400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.397683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.397958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.398701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:18.399008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.400390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ea00d-bec8-4e33-971d-3ca410329a09 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:18.402592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:18.403529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:18.404458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 326/639] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:18.404954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.412294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 314/640] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:18.413675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.414041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.414328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.431555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:18.432774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:18.476728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.500149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 315/641] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:18.500972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.509702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfee817-7e9e-43ab-b7b7-4284308733ec None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.546239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.547172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.547455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.547731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/8b3c45809bac4828bcde30494ab7a0d2/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.561789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=8b3c45809bac4828bcde30494ab7a0d2, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.563250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.576468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 327/642] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:18 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/8b3c45809bac4828bcde30494ab7a0d2/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:18.577272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.608914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61e38f-d112-49e9-a60d-e11277409748 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.617183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.651087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.651892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.652207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.652591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.665046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.666542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:18.685655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:18.686699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:18.687583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:18.687583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:18.687583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:18.687583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:18.687583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 316/643] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:18.687872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.699575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4604d-4591-483f-9b60-195ba58c93f4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.721231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.744488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.745093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.745376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.745654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.746881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:18.747197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.748985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.761831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 328/644] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:18.762648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.802845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b60d9b-23bf-4142-aa90-3ca34fd7a718 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.838095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.838885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.839160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.839469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.850663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:18.852127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:18.870210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:18.871198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:18.872756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:18.872756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:18.872756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:18.872756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:18.872756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 317/645] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:18.872756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.874059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.883280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03769c9-1a8b-43a6-9e2d-4f617c0e6a7e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:18.919450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:18.920595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:18.921031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:18.921318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:18.932575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:18.940516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:18.941636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:18.942696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:18.946805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 329/646] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:18.947660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:18.962890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:18.982282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a20f8-9bdd-490f-92a4-49da3ff7a99c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.019089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.019677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.019961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.020823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.021698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:19.022012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.023456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.040365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 318/647] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:19.040365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.056628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22d50f7-f3a7-48b8-903f-8b3c34efc581 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.090401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.091149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.091494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.091841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.092775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=37f8217df17a44a989ce1c7ab8a9ac1a) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:19.093153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.094664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.108725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 330/648] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:18 2022] GET /identity/v3/role_assignments?user.id=37f8217df17a44a989ce1c7ab8a9ac1a&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:19.108725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.135841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.196299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc46990-8583-4c6e-a68f-dfaac5718a20 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.233090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76374a-c178-435e-9623-b656e12d2ccd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.251636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.253120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.253649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.254084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.255583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:19.256049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.258332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.274147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.274896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.275274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.275554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.276345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:19.276652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.277929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 319/649] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:19.278056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.279118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.282895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.291718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 331/650] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:19.292023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.400539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.417624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e5b1c-be38-4aaf-8a4b-21e387c18178 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.421457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7284ec-4dcf-40b5-84cc-896c9298d6dd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.455866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.457013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.457303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.457583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.462451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.463439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.463879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.464334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.467002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.469016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.473234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.475396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:19.485484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:19.486629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:19.487611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 332/651] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:19.488275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:19.494765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:19.495875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:19.496861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:19.500416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 320/652] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:19.501968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.580902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.605348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e973e-9694-4411-93a8-ce2819df786b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.625443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e794b7-a262-4303-bebe-b7ee0758e796 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.642823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.644196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.644836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.645477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.660390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.662258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.663630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.664342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.664867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.664976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.665201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.666832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:19.667149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.668612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:19.687291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:19.688265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:19.689144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:19.689144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:19.689144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:19.689144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:19.689361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 321/654] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:19.690369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.693578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 333/654] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:19.694960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.808381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ff5ef-24ea-420a-be4d-28b53f38d95e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.834696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40b458-fe29-4e3f-a0ac-f176a036f56c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:19.845327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.845973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.846255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.846636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.861460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.886440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:19.887506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:19.888279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:19.889626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:19.890672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:19.891195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.892849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:19.917056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:19.919952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 322/655] 10.222.0.17 () {56 vars in 1137 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:19.921145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:19.927134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:19.928120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.043635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 334/656] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:12:20.045524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.047774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4cd5a-d14c-4780-8f5c-115979e3e095 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.087753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.088733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.089242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.089708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.090622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:20.091106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.092688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.106157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 323/657] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:20.107282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.149303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.157709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.165772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87944c6-a8b5-4a17-a2c0-48bda0d2a649 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.202443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.203348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.203348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.203630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.216229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd9a2f-8b1a-49b2-a7fb-3be62e3ff153 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.253203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.254387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.254903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.255384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.267392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.269301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.284622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.286683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:20.289510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:20.290372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:20.291237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:20.291237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:20.291237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:20.291237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:20.297768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 324/658] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:20.298916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.388112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.400683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 335/659] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:20.400683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.412153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.424831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a69909-26b5-4f98-8de2-ccf104ab9ce2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.462356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.463588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.464183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.464774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.473265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.475544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:20.490607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:20.491571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 15 19:12:20.492543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:20.495212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 325/660] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:20.496252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.512603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743141c-82e4-4ac7-8cdb-fc69f8e15fee None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.552923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.553906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.554020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.554276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.555440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:20.555917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.557748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.574914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 336/661] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:20.574914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.610570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c92ebd-c75e-4ede-9cf4-93f52a0a1f3c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.631923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.650171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.651643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.652160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.652641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.665787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.667533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.680025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e056f0a-6b31-4345-8699-92f02349677b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:20.687649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:20.688664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:20.694766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:20.694766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:20.694766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:20.694766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:20.694766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 326/662] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:20.694766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.696374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.716843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.717513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.717841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.718615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.719934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:20.720272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.722529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447bd48-6385-46be-8cca-5f3d82944796 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.751840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 337/663] 10.222.0.17 () {56 vars in 1139 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:20.751840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd4f8-89e9-4aae-b34a-087b5f2bdf20 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.751840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd4f8-89e9-4aae-b34a-087b5f2bdf20 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.751840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6fd4f8-89e9-4aae-b34a-087b5f2bdf20 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.753021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 338/664] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:20.753985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd4f8-89e9-4aae-b34a-087b5f2bdf20 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.809838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686deb4-2106-48df-88ff-d239876a199c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.846848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.847755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.848037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.848367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.855446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.857048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.864064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6fd4f8-89e9-4aae-b34a-087b5f2bdf20 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:20.872268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:20.873150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:20.874300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 327/665] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:20.875327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:20.881632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.907930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:20.908549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.908870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.909155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.909933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:20.910245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:20.911728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd341b23-85c7-43f0-a2db-b4f0af5c09db None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:20.929404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 339/666] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:20.929404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:20.929404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:20.929404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:20.943584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:20.948158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:20.949352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:20.988321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa71c2-2979-4384-9b7a-7947044958f5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.022679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 340/667] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:21.023906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.028819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.029759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.030051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.030383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.103001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.104508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.140468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db10d1-19f0-4670-99a1-3d7e8915a7ca None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.142382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.167017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.182790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.183454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.183740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.184019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.184840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:21.185269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.186848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.204036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 341/668] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:21.204036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.215772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 328/669] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:21.220990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.310263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413fb77-aff8-4604-917f-75f2474bdd0f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.326432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c21fd-6cd8-44c2-81af-cf2374bf69c1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.343301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.344128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.344420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.344701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.352368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.353857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.360721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.361503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.361784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.362058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:21.368491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:21.369501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:21.370496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 342/670] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:21.371104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.375202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:21.383252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:21.384199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:21.385338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 329/671] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:21.386400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.397558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.414305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.471863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb3050-18e0-4fa6-a360-8c7a3570f0bc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.489451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1653c726-0b31-4153-b187-70195831ca72 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.505767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.506230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.506512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.506789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.507615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=6802452aab2f43d4b965be689758b611) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:21.507926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.509400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998ae6-a1e1-423c-9217-8903c0616430 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.515119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 343/672] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/role_assignments?user.id=6802452aab2f43d4b965be689758b611&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:21.516403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1da91-aec0-445f-aa5e-4157f705c400 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.516696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1da91-aec0-445f-aa5e-4157f705c400 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.516979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d1da91-aec0-445f-aa5e-4157f705c400 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.518364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 344/673] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:21.518960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1da91-aec0-445f-aa5e-4157f705c400 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.528509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.528509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.528834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.529069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.542639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.544182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:21.565465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:21.566435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:21.567965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:21.567965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:21.567965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:21.567965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:21.567965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 330/674] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:21.569131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.625007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.646015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d1da91-aec0-445f-aa5e-4157f705c400 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.653345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.685117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.685787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.686061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.686376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.687113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:21.687973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.688917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287603eb-8eb5-4ec6-b5e5-e1eb2a178065 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.689042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d0def-d6a9-4827-a687-881658cbbe05 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.701756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 345/675] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:21.702813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.703398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.703398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.720239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:21.721440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:21.727329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.728432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.728432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.728600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.730171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:21.730171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.731447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.740122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 331/676] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:21.740726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.805914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 346/677] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:21.806895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.859538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1e5f1-d2f7-42f6-946d-c55fe6d94ae4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.893784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.894498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.895726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.896010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.896315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.897118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:21.897454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.898826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.901246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:21.912223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 332/678] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:21.913127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:21.928298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184732c-08ed-41ce-88a7-f521f38c77ab None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:21.965513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:21.966460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:21.966931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:21.967358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:21.968366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=43fda9627eb84093bb59ceb1a553baf7) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:21.968834 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:21.970757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:21.978046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 347/679] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/role_assignments?user.id=43fda9627eb84093bb59ceb1a553baf7&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:21.978989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.033004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b433398-c102-436a-8f23-e149feb458b8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.081214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.082413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.082413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.083157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.094676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.096190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:22.120266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:22.121225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:22.122198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:22.122198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:22.122198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:22.122198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:22.124238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44c90e-7375-44d0-ba32-28cc03190c37 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.126674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 333/680] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:22.128123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.172967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.173769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.174043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.174315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.185496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:22.186779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:22.192947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:22.198384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:22.199323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:22.200155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 348/681] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:22.200846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.262437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1bc2d-322b-4d31-8fc5-aad51be3dac8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.309453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.310856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.311596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.312388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.323641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e783af2-a7a0-4ca7-8085-6f95343ce975 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.325011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.327816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:22.347747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:22.348906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:22.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:22.352306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 334/682] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:22.353316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.362016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.362679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.362959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.363233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.364031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:22.364370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.365871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.384223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 349/683] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:22.384223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.469555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:22.478433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad48964-a2a6-4f7b-b1ca-a91723c1cc95 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.488363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:22.518238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.519775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.520939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.521636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.522845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:22.523347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.525582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.526556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce60da-9595-4396-af5d-59cfdfd8d757 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.537156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 335/684] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:22.538243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.573994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.574882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.575234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.575579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.576715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:22.577948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.579480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.595395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 350/685] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:22.596341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.656489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f359bd-2ca4-4fec-a94f-919ef246edb4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.700323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.701228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.701586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.702078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.714825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.716788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.718135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d9803-5a50-4384-b603-887ec11ed794 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:22.733154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:22.734158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:22.735106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 336/686] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:22.735728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.743424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:22.758985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.760182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.760689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.761170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.773147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.775551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.779236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:22.793865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:22.794801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:22.795669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:22.795669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:22.795669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:22.795669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:22.797902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 351/687] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:22.798906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.863126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26339d4-0b96-4145-bd29-52d8f22eed21 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.901714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.902368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.902650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.902927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.903759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=9f135ce75fbc4b12977cc8f009083ff9) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:22.904066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.904522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce4e92c-a25f-48ae-9ed3-e4801c06863b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:22.905573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.912539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 337/688] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/role_assignments?user.id=9f135ce75fbc4b12977cc8f009083ff9&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:22.913348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:22.942713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:22.944107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:22.944599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:22.945109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:22.946183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:22.946724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:22.948457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:22.968553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 352/689] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:22.981321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:23.025039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:23.029359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb04287-c069-4fca-b9f1-04a175813129 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.034081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:23.070952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.071693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.071981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.072288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.073271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:23.073584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:23.075225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:23.097533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 338/690] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:23.104687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:23.120885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9db4146-94dd-44e8-b22c-5e222dfe15db None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.159911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.161729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.162025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.162456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.272296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:23.273796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:23.300410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebc616-b434-4138-a63b-faa8a74f087a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.347161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:23.357060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.357939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.358220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.358493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.370255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:23.372252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:23.391847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:23.393353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:23.399283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 339/691] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:12:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:23.404285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:23.443797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:23.525839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 353/692] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:23.530751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:23.647334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21033774-682c-4dcc-a494-fa7c87646629 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.661970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbd6fca-67aa-4d77-991f-0adb4564af17 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.694359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:23.702223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.703333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.703820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.704382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.719292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:23.721699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:23.737956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.738647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.739129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.739343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.740258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:23.740573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:23.742037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594941b-0207-4a7b-a9e8-3eaa5bd49ba5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:23.753390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:23.754878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:23.756185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:23.756185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:23.756185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:23.756185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:23.758512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 354/693] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:23.764400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:23.789182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 340/694] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:12:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:23.790225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ba33-0948-4e77-8d52-d6d066820362 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.790512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ba33-0948-4e77-8d52-d6d066820362 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.790796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901ba33-0948-4e77-8d52-d6d066820362 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.792010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 341/695] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:23.797430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901ba33-0948-4e77-8d52-d6d066820362 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:23.890723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:23.904689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47158769-1e77-4ebf-a913-b82a87b6b1f2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.930046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901ba33-0948-4e77-8d52-d6d066820362 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:23.954376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625235f-613b-4c63-a621-065e93f535de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.955903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625235f-613b-4c63-a621-065e93f535de None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.956510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625235f-613b-4c63-a621-065e93f535de None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.957010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625235f-613b-4c63-a621-065e93f535de None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.959075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625235f-613b-4c63-a621-065e93f535de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:23.959944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625235f-613b-4c63-a621-065e93f535de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:23.962857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625235f-613b-4c63-a621-065e93f535de None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:23.971367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:23.972492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.972492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.972691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:23.987832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 355/696] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:23.990138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:23.990606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:23.991034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.017408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:24.019770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:24.058682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.060280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.064705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.108254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 356/697] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:24.109920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.157221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.195584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 342/698] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 15 19:12:24.196520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.220035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb699-3126-40a1-b6f9-3e926e4c835c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.257590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.258661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.259116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.259870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.267411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.269524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:24.286396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:24.287609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:24.288525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:24.296606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 357/699] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:24.298740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.306067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c240-1660-4699-a5b1-5b5ff457530b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.334285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.341581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.342373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.342655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.342928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.356265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.356265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:24.375819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:24.376854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:24.377740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:24.377740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:24.377740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:24.377740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:24.382243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 343/700] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:24.383140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.409136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.420750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec2946e-aeab-47fd-ba24-1c5ae8c8c850 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.458318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.459553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.460015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.460460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.472486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.475069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.489456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac7d49-ef54-4475-9145-4e16756cc2ad None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:24.494482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:24.495485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:24.496403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:24.496403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:24.496403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:24.496403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:24.497765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 358/701] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:24.497765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.529583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.530243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.530547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.530802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.531584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:24.531890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.533341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e66d87-35c7-4a46-b7bf-98ca36656fad None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.545479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 344/702] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:24.547131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ed40-08be-4820-b135-fc31e333d6c7 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.547419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ed40-08be-4820-b135-fc31e333d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.547694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3989ed40-08be-4820-b135-fc31e333d6c7 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.552951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 345/703] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:24.552951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989ed40-08be-4820-b135-fc31e333d6c7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.602900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.622274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76b52da-5e2a-4ae7-ac61-fce0bdca80fe None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.639716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.657201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3989ed40-08be-4820-b135-fc31e333d6c7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.660101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.660234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.660480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.661242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.661968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:24.662267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.663639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.680110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 359/704] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:24.680110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.700547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.701219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.701508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.701789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.702521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:24.702857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.704240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c2881-8de6-447d-bdc6-33f80081235f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.721137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 346/705] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:24.721137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.721137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.721137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.740052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:24.741292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:24.800981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f74f30-5303-4ee0-9316-8f3ba2c53afa None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.820314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 347/706] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:24.820314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.851019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.851683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.851962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.852434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.853104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=8b3c45809bac4828bcde30494ab7a0d2) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:24.853427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.854938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:24.861526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 360/707] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/role_assignments?user.id=8b3c45809bac4828bcde30494ab7a0d2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:24.862149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:24.876536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.912694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:24.943139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b7944-fece-4d79-bb92-846d231c251c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:24.982184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:24.983028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:24.983312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:24.983591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:24.984357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:24.985857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.015884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52472210-5a10-4054-a849-5cd29be8b32b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.016631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 348/708] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:12:24 2022] POST /identity/v3/users => generated 254 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:25.017528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:25.065068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.065971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.066259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.066685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.075537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.077283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:25.094988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:25.095964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:25.097210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 361/709] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:25.102115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:25.143190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04255471-cbce-4cfc-acf1-70860061f002 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.174963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:25.183668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.184726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.185187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.185471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.186330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.186754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:25.187716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:25.204496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:25.206816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:25.208857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 349/710] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:25.212671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:25.227982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19f99e-2401-4b82-ac9d-9d7e917a4bc1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.272415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.273526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.274023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.274511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.275623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:25.276318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.277992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.294392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 362/711] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:25.301153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:25.334088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a4d5a2-cf7f-4abb-abf1-eac85e299451 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.372719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.373637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.373922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.374201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.374911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:25.375214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.376633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537dd6d-1bc8-4f3e-b4bd-f6db07b6b998 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.386793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 350/712] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:25.389324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebf59b-4649-41e3-b736-045ebb9562fc None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.389782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebf59b-4649-41e3-b736-045ebb9562fc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.390245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebf59b-4649-41e3-b736-045ebb9562fc None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.394123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 351/713] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:25.395674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebf59b-4649-41e3-b736-045ebb9562fc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:25.429296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bd7fbc-d67f-4a5d-a543-f42e3fba1509 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.458519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:25.467973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.468942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.469442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.469913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.470941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=37f8217df17a44a989ce1c7ab8a9ac1a) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:25.471481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.473736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51e1b3-a77e-46f5-aabe-ebc41eb39432 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.480289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 363/714] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/role_assignments?user.id=37f8217df17a44a989ce1c7ab8a9ac1a&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:25.481817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.482313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.482812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.507849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:25.509474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:25.511389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:25.613554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebf59b-4649-41e3-b736-045ebb9562fc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.627110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 364/715] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:25.654460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:25.679963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.682280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.682736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.684029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.684946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:25.688375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.689983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.704111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 352/716] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:25.851690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238dfa2-a2ff-4a60-be9b-d237eeb449ea None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:25.896899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:25.903576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:25.904443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:25.904760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:25.905032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] PATH_INFO: `/v3/services/network` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:25.905700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:25.907183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:25.930724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:25.931719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:25.933253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 365/717] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/services/network => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:25.937741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:26.046634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9202b6-8ddf-44c2-829a-ca5c7ed35fb0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:26.071745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:26.086861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:26.087534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:26.087816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:26.088113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:26.088830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] RBAC: Adding query filter params (name=network) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:26.089139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:26.090525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1dd54c-f00b-47fd-9079-546b97bb099b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:26.095953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 353/718] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:26.102448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:26.221719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a68ee-2c2c-4306-bee9-1cbb1a4980db None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:26.260584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:26.261255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:26.261543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:26.261815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:26.262528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] RBAC: Adding query filter params (type=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:26.262829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:26.264236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b1087-304e-4037-922f-fa34982b6046 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:26.271091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 366/719] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:27.474015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04030-ae77-459d-8b18-ce44da7c215a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:27.475201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04030-ae77-459d-8b18-ce44da7c215a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:27.475491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b04030-ae77-459d-8b18-ce44da7c215a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:27.477188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 354/720] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:27.482813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:27.483674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:27.484218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:27.506479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:27.507784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:27.568491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:27.569966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb37b3-e3a3-4d7e-92a8-056a1a3d6d37 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:27.570255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb37b3-e3a3-4d7e-92a8-056a1a3d6d37 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:27.570533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb37b3-e3a3-4d7e-92a8-056a1a3d6d37 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:27.572115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 355/721] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:27.577290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:27.577835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:27.578342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:27.592852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 367/722] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:27.609603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:27.611143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:27.660051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:27.742370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 356/723] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:27.798474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:27.804192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ffac7-f4b9-49ef-b337-f74e966ec57b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:27.882382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:27.883617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:27.884129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:27.884612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:27.902090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:27.923253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:27.924315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:27.925182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 368/724] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:27.932849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:27.960720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1207a02-c553-4698-afc6-5dba28a1f713 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:27.988444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:27.997120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.007735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7605fc9-2d2a-4fbe-a316-d00d7a469f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.009065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7605fc9-2d2a-4fbe-a316-d00d7a469f47 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.009844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7605fc9-2d2a-4fbe-a316-d00d7a469f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.010395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7605fc9-2d2a-4fbe-a316-d00d7a469f47 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.024723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7605fc9-2d2a-4fbe-a316-d00d7a469f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c7605fc9-2d2a-4fbe-a316-d00d7a469f47 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:28.034779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:28.036525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:28.037546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:28.040762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 357/725] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:28.042871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522551c3-fa41-4175-bd3c-b7ca3380697f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.043511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522551c3-fa41-4175-bd3c-b7ca3380697f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.044071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522551c3-fa41-4175-bd3c-b7ca3380697f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.045928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 358/726] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:28.046792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522551c3-fa41-4175-bd3c-b7ca3380697f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.068936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed11b0ee-a561-4b85-b4e2-789ea39a6b65 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.106006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.106998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.107457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.108007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.109052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:28.109531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.111082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06b033-9f29-4d88-b94c-5b1a9cb35224 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.120784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 369/727] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:28.122237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.122735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.123452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.140470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:28.141929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:28.173503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522551c3-fa41-4175-bd3c-b7ca3380697f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.215804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.216614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.216968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.217319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.218164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:28.218346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 370/728] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:28.219324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.224765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.226287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.241694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 359/729] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:28.246936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.252925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.301937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.326497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea95684-ed9b-4949-a90f-8dcabd2ba649 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.363083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.364104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.364555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.364833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.368800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b665a-9d7b-4e77-8ec5-1dcf92cf1ba8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.375614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.377102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-23b5bc99-2e27-4374-b6b4-e362bb0a5e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:28.395859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:28.396939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:28.397797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:28.397797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:28.397797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:28.397797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:28.397797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 371/730] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:28.398449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e224-6e97-45bb-bfc8-20ec7a49aa9c None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.398604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e224-6e97-45bb-bfc8-20ec7a49aa9c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.398872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b8e224-6e97-45bb-bfc8-20ec7a49aa9c None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.400250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 372/731] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:28.400841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e224-6e97-45bb-bfc8-20ec7a49aa9c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.415516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.417593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.418069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.418486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.430242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.431796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:28.450208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:28.451168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:28.452054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:28.452054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:28.452054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:28.452054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:28.452054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 360/732] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:28.452779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.499869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.520412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b8e224-6e97-45bb-bfc8-20ec7a49aa9c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.550527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.558393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e585c1-a73e-4679-afb4-25e8c7abc625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.559191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e585c1-a73e-4679-afb4-25e8c7abc625 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.559470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e585c1-a73e-4679-afb4-25e8c7abc625 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.559745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e585c1-a73e-4679-afb4-25e8c7abc625 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.561787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ac83-e592-4bdd-9e2b-a4f060d57a42 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.571271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e585c1-a73e-4679-afb4-25e8c7abc625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-76e585c1-a73e-4679-afb4-25e8c7abc625 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:28.579340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:28.580282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:28.581139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 373/733] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:28.582987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.582987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.582987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.601924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:28.603123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:28.605775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.606452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.606739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.607024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.607751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:28.608062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.609478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.627626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 361/734] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:28.627626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.687557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 374/735] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:28.688550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.740781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714c643-b33a-43e3-b65c-6361fa143945 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.780216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.780892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.781178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.781456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.782195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:28.782505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.783903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.784179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.802203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 362/736] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:28.802203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.813957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:28.818556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61ecc4-0de9-4efe-b04e-f10db5b35295 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.856429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.857097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.857488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.857765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.858624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:28.858932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.860447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.874232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 375/737] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:28.874232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.906427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b94089-1974-417c-99eb-258bf0c0adb3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:28.941369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:28.942279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:28.942558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:28.942839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:28.949935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:28.951376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:28.966898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:28.967851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 363/738] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:28.968820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:28.977993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6628ca07-565b-49ce-894e-97faaa0a7b94 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.012658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.013308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.013588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.013863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.014572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.015931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.019630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.032839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 376/739] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:12:28 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:29.033558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.040957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.077651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb5ceb1-1dd6-4736-9025-bea206a51419 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.131262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.132095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.132386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.132667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.141280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.142776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.153541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bb81f-d472-4dca-ac4a-a14b8d36319c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e2968578-c0a9-4ce7-910a-59bda5ea5b3d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:29.159167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:29.160504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:29.161448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:29.165352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 364/740] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:29.166654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a44a36-bc5e-4507-b719-dc22507120fa None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.166943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a44a36-bc5e-4507-b719-dc22507120fa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.167220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a44a36-bc5e-4507-b719-dc22507120fa None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.168617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 365/741] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:29.169325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a44a36-bc5e-4507-b719-dc22507120fa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.200197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.201004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.201289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.201569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.215428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.215699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:29.234639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:29.235745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:29.237061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:29.237061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:29.237061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:29.237061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:29.237061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 377/742] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:29.237061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.290686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.293920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a44a36-bc5e-4507-b719-dc22507120fa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.324126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.334779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.335453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.335738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.336037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.336885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:29.337191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.338616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c701033-eb5d-49c3-8255-aeeec08e47b3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.347391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d49a8-1adc-4a2a-80bd-d017688a0ab8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.361579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 366/743] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:29.361579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.361579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.361579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.378945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:29.380230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:29.383079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.383759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.384041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.384348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.385194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:29.385503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.386933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.411511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 378/744] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:29.411511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.455077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 367/745] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 15 19:12:29.455808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.534682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cf929-9366-4cfb-b649-d99fa8d3b624 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.543773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.572258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d1746-c79c-4395-bc5f-2d9f632130a8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.574121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.574906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.575184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.575455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.576196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:29.576500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.577874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.587787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.592139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 379/746] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:29.597266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.610102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.610760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.611050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.611332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.682879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.684385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.701941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa69d9f2-2549-4127-87e4-642bf3226a78 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.740252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.740921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.741209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.741490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.778324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.809551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 368/747] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:29.810380 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.816379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.817889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.825944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:29.921111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a3b39c-65d5-4d5d-964e-b0260be56924 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:29.927765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 380/748] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 15 19:12:29.927765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:29.960936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:29.961695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:29.961981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:29.962258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:29.962976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:29.964391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:29.988115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 369/749] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:12:29 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:29.988115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.006316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.053987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70baf518-ef9c-457a-8a7d-73d105060452 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.080763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.114380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.115186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.115293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b376206-9b3c-4250-b4f3-087603160059 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.115462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.115743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.126988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.129447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:30.152378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:30.153378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:30.154312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 381/750] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:30.155615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.162182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.163015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.163301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.163577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.181314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.183089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:30.203511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:30.204461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:30.205358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:30.205358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:30.205358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:30.205358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:30.209147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 370/751] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:30.209147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.293064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278202f0-d981-4a43-87a8-fc14c4464045 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.333100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c450b25-44de-4fb6-8734-623baa338603 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.333419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.349022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.350148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.350763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.351273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.360647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.363749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.366883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.370408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.370842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.371126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.371404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.372300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:30.372618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.374443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:30.388256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:30.389179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:30.390049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:30.390049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:30.390049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:30.390049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:30.396886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 371/752] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:30.396886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.397112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 382/753] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:30.398606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.502447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d711e-1c19-430d-81db-5b3ed6b13c39 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.513820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e39ac-8947-42fb-b06f-101a867d8174 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.539515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.540407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.540794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.541177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.542091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:30.542395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.543858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.555839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.557216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.557666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.558154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.561782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 372/754] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:30.562585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.570363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.572198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:30.591745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:30.592795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:30.593873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:30.593873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:30.593873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:30.593873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:30.595874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 383/755] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:30.597132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.612862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.629937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.673671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20187e5-4429-49d4-bd85-0fbd430881ef None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.710993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.711415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dc518c-7d76-4b88-aa6c-ea484cf3d555 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.711930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.712334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.712703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.713629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:30.714037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.715624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.729790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 373/756] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:30.731510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.759800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.760889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.761440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.761857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.772726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.775016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:30.791381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:30.792364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:30.793248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:30.795937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 384/757] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:30.797006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.848511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd079a-6614-4e84-848d-91e276142602 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.850846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.887261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.888232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.888610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.888954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.889845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:30.890219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.891676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.893741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:30.905867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 374/758] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:30.909970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:30.914514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041da2a0-7ada-42a5-8214-7ef5fc4f9fbd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:30.950788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:30.951974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:30.952497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:30.952979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:30.961847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:30.963611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:30.979643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:30.980700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:30.981736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:30.983721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 385/759] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:30.984754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.021067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61290054-1c33-4525-9c09-1fdf93a93518 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.058290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.059003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.059281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.059609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.060424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:31.060730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.062879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.075890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 375/760] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:31.076745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.100388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29bb5e4-1ec3-43fd-9f6e-b34be4d26e60 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.101678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:31.140048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.141337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.141835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.142426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.152050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.154798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8f52ab53-f21a-4641-81fb-4789f9248776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:31.171186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:31.172161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 386/761] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb96e4f-6dda-419d-b189-647852a4c025 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.179192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb96e4f-6dda-419d-b189-647852a4c025 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.179783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb96e4f-6dda-419d-b189-647852a4c025 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.179783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 387/762] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:31.179783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb96e4f-6dda-419d-b189-647852a4c025 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.179972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1c759-8ba2-47ea-abcd-2f626e55e47d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.190203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:31.217177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.217862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.218147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.218426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.219235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:31.219673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.221303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.235880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 376/763] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:31.236804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.300544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb96e4f-6dda-419d-b189-647852a4c025 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.336407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.337063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.337330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:31.337451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.337595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.338420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=62f4fa05f65c4643898254268882b8f2) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:31.338726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.340207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a93e2-6761-4bfc-b549-6e6e322721a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.345934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 388/764] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:31 2022] GET /identity/v3/role_assignments?user.id=62f4fa05f65c4643898254268882b8f2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:31.347102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.347384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.347659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.366113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:31.367607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:31.369112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e8a607-9ceb-4d79-bedc-0ac11f4086c8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.418983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.419768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.420279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.420751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.421782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:31.422806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.424851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.447670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 377/765] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:31.447670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.453015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:31.459355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 389/766] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:31.460743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.580317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c338843-70d2-497f-a9da-82e053854bf4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.602139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f7c36f-e5d1-4ab5-849a-27e58800427c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.620326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.620951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.621190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.621467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.622361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=8b3c45809bac4828bcde30494ab7a0d2) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:31.622613 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.624333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.636363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 378/767] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:31 2022] GET /identity/v3/role_assignments?user.id=8b3c45809bac4828bcde30494ab7a0d2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:31.637238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.650444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.651727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.652245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.653431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/37f8217df17a44a989ce1c7ab8a9ac1a/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.656245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=37f8217df17a44a989ce1c7ab8a9ac1a, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.657903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:31.659474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.678443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 390/768] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:31 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/37f8217df17a44a989ce1c7ab8a9ac1a/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:31.819255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01cd4e-19d1-49f9-b7c0-d727acbc9701 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:31.867872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:31.868807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:31.869114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:31.869395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:31.870069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:31.871544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:31.889609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:31.890587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:31.891452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:31.893219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 379/769] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Jul 15 19:12:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:31.896881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:31.964419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:32.020552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b9338-0e2f-46de-8fb1-9021de584194 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:32.071178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:32.071851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.072158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.072439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.073228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:32.073535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:32.074923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16697e-5a7a-41b0-aebe-94312222a91c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:32.084478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 391/770] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Jul 15 19:12:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:32.090208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:32.226606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:32.273117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386c3c-e6d0-4126-8c86-a57f9a3e52a3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.273443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386c3c-e6d0-4126-8c86-a57f9a3e52a3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.274115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24386c3c-e6d0-4126-8c86-a57f9a3e52a3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.275310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 392/771] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:32.280735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b84b8-faf8-4dbd-9fb8-649db04dddae None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:32.293583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.293885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.294219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.329461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:32.331218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.331506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.331784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.333069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:32.333381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:32.334776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:32.338196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:32.339402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:32.351689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 380/772] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Jul 15 19:12:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:32.359023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:32.420903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 393/773] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:32.465399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:32.549928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a787ce-812c-4a5e-9395-f22859c4d89a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:32.577465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:32.613081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:32.614288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.614575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.614853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.615644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] RBAC: Adding query filter params (interface=public, service_id=4fc252d0f1284307837ef70dd35b9a98, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:32.615957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:32.617361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:32.624680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:32.637627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 381/774] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:12:32 2022] GET /identity/v3/endpoints?service_id=4fc252d0f1284307837ef70dd35b9a98&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:32.642522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:32.657540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235f0ab-663f-4fca-ab5e-edb3677f2cf4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:32.721464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:32.722272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.722556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.722893 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.723563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:32.724975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:32.746179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:32.747181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:32.747995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:32.760365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 394/775] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:32 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:32.761256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:32.831504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec56b3-74a2-4411-aec1-294749f06810 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:32.896631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:32.897298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:32.897582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:32.897857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:32.898540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:32.899912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:32.916649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 382/776] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:32 2022] GET /identity/v3/services => generated 772 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:32.950095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3160a25-8664-4ed9-ae13-8fd743c97bc0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:32.976652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:33.006351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:33.007036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:33.007319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:33.007598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:33.008376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] RBAC: Adding query filter params (name=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:33.008679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:33.010072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:33.021902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 395/777] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:33.027509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:33.120504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:33.174030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29dda88-fea6-4222-9e66-526b0b5da979 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:33.210384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:33.211060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:33.211340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:33.211609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:33.212358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] RBAC: Adding query filter params (type=network) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:33.212663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:33.214029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:33.219711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 383/778] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:33.229277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:33.342129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd52fc5a-83a4-4ff1-be47-ad4925e2cb04 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:33.378850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:33.379783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:33.380171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:33.380498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:33.381309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] RBAC: Adding query filter params (interface=public, service_id=9043cf39452d47498cdfbdb9c9b6f7f8, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:33.381622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:33.383001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226369f-a80f-4b4f-82e7-539d25bc3a5c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:33.391279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 396/779] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:12:33 2022] GET /identity/v3/endpoints?service_id=9043cf39452d47498cdfbdb9c9b6f7f8&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:33.397530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:33.522337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc64395-f1f8-4653-ae7a-adccb447544c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:33.558930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:33.573752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd0c9a-0e6b-4018-929c-6f3d8a2c85f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:33.574432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd0c9a-0e6b-4018-929c-6f3d8a2c85f2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:33.574714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd0c9a-0e6b-4018-929c-6f3d8a2c85f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:33.574989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd0c9a-0e6b-4018-929c-6f3d8a2c85f2 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:33.575678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd0c9a-0e6b-4018-929c-6f3d8a2c85f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:33.577082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd0c9a-0e6b-4018-929c-6f3d8a2c85f2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:33.588122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 384/780] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:33 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:33.814709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41354-e07c-45b9-8969-e2db586983ad None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:33.815220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41354-e07c-45b9-8969-e2db586983ad None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:33.815298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c41354-e07c-45b9-8969-e2db586983ad None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:33.817429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 397/781] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:33.822944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d8f1-e506-47ac-a562-50e96f9059ce None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:33.823355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d8f1-e506-47ac-a562-50e96f9059ce None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:33.823529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d8f1-e506-47ac-a562-50e96f9059ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:33.840706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-70e4d8f1-e506-47ac-a562-50e96f9059ce None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:33.841920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-70e4d8f1-e506-47ac-a562-50e96f9059ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:33.916195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 385/782] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:33.938904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c41354-e07c-45b9-8969-e2db586983ad None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.011514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.066657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335871c-f55b-4817-8cb9-769280a5c5b9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.066861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335871c-f55b-4817-8cb9-769280a5c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.067361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8335871c-f55b-4817-8cb9-769280a5c5b9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.070383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 386/783] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:34.074778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dff48e-11ef-437d-b293-5833fe8cb9e4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.075278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dff48e-11ef-437d-b293-5833fe8cb9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.075742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dff48e-11ef-437d-b293-5833fe8cb9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.095096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c41354-e07c-45b9-8969-e2db586983ad None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.109470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-20dff48e-11ef-437d-b293-5833fe8cb9e4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:34.110665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-20dff48e-11ef-437d-b293-5833fe8cb9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:34.136910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5ea62b-8d4c-499b-ab64-9d56a682e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.137955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ea62b-8d4c-499b-ab64-9d56a682e286 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.138382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ea62b-8d4c-499b-ab64-9d56a682e286 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.138824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5ea62b-8d4c-499b-ab64-9d56a682e286 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.150476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5ea62b-8d4c-499b-ab64-9d56a682e286 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3d5ea62b-8d4c-499b-ab64-9d56a682e286 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:34.158634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:34.159728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:34.161314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:34.163559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 398/784] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:34.165074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2405df-8805-4ef7-a4ca-43ba2a7b11f5 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.165502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2405df-8805-4ef7-a4ca-43ba2a7b11f5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.165922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2405df-8805-4ef7-a4ca-43ba2a7b11f5 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.167545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 399/785] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:34.172341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2405df-8805-4ef7-a4ca-43ba2a7b11f5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.183436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 387/786] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:34.184880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.185169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.185446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.203512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:34.204722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:34.273435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.285928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2405df-8805-4ef7-a4ca-43ba2a7b11f5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.291136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 388/787] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:34.296805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.297161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.344236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.345506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.346012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.347347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.349062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:34.349659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:34.351298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:34.369067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 400/788] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:34.369067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.441834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97662eb0-0ca1-49ac-bfb3-ffe53f292aa8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.478313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca8de-254b-4f99-acc5-bc099e52b087 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.485588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.486676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.487163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.487656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.499382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:34.508029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:34.509086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:34.509916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:34.512578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 389/789] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:34.513589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.521874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.522946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.523430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.523960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.535691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:34.543937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:34.544872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:34.545717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:34.548460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 401/790] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:34.549462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.562647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.587740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.630995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9cf70-3804-443a-b6d3-61dcf908a39a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.668112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.668949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.669304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.669796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.688583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a535b5-47b5-4ecd-9e03-8b6a8a56c2dc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.692722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:34.693707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:34.711829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:34.712772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:34.713636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:34.713636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:34.713636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:34.713636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:34.713636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 390/791] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:34.714408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.735013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.735983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.736543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.737692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.739951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:34.740495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:34.742997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:34.756903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 402/792] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:34.758065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.826850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.834566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354c9a-291f-4a39-9c47-614a4889155c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.871806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.872528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.872914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.873552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.874408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:34.874818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:34.876415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:34.886301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 391/793] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:34.887393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:34.900493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:34.904322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388441c-fb81-42ef-86ae-696f18837d31 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:34.946559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:34.947577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:34.947950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:34.948355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:34.949209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:34.949617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:34.951299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:34.965981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 403/794] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:34.967073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.029148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c010ce9-870b-4d2a-94a3-1984a01ccc6e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.070597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.071639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.072125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.072664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.076383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e8a1b-5972-45db-8e6e-109cc142871a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.086019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.087790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:35.107593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:35.108599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:35.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:35.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:35.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:35.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:35.111447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 392/795] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:35.112404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.117640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.118551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.118937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.119317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.126411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.131677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.133346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:35.154873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:35.156769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:35.157756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:35.157756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:35.157756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:35.157756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:35.158943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 404/796] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:35.159865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.163274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.244772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3054cc-57c7-4ca1-b80a-715e2260d7a4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.271422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906210fb-f0f6-47ba-9cb5-9e3ebc904a1f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.284375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.286182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.286664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.287122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.295674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.297462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.307407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.308233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.308591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.308968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.309766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:35.310140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.311616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:35.315352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:35.316878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:35.318461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:35.320998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 393/797] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:35.322048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.328392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 405/798] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:35.329426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.399605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.401801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.440507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b4d5d-c292-49ca-bda9-9e3e1e2c6a68 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.447665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb78568-b146-4a1a-93e0-e46254f82898 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.480645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.481504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.481892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.482294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.483129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:35.483533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.485041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.485432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.486255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.486785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.487202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.488891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:35.489415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.491077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.500717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 394/799] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:35.502532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 406/800] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:35.513138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.624317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd349a4-8568-4c6a-b63d-ac3403cae8a0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.627794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f185eb-f92e-4519-81f5-1d6816bfa659 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.646936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.655982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.663870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.664340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.664707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.664994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.665109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.665266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.665420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.665662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.672987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.673106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.674470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.674571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:35.689945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:35.690978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:12:35.693619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-48c83d4b-12a2-43a6-a6e9-2278f389a424 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:35.694187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:35.695022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:35.695938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:35.696491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 395/801] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:35.696491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.701289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 407/802] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:35.702695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440088d-dde6-4329-9ca2-53626eb19a1c None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.702979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440088d-dde6-4329-9ca2-53626eb19a1c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.703252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440088d-dde6-4329-9ca2-53626eb19a1c None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.704767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 408/803] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:35.709613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440088d-dde6-4329-9ca2-53626eb19a1c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.809924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f7a5a-df9d-4e5f-871f-ad5aa175827c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.829295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440088d-dde6-4329-9ca2-53626eb19a1c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:35.848894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.849883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.850160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.850434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.861178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.862644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.865530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:35.866503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.866784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.867058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.867896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:35.868241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:35.869660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12ae1e-d323-4325-ad8b-4df217655e57 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:35.881011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:35.881918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:35.882808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:35.882808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:35.882808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:35.882808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:35.882808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 396/804] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:35.883091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.888213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 409/805] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:35.889360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61262e3-0ac2-465f-809b-939351afdf68 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.889648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61262e3-0ac2-465f-809b-939351afdf68 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.889924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61262e3-0ac2-465f-809b-939351afdf68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.905393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b61262e3-0ac2-465f-809b-939351afdf68 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:35.906580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b61262e3-0ac2-465f-809b-939351afdf68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:35.926356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.936712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:35.988841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 410/806] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:35.988841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d70037-e337-4d3c-bf5a-602173750c12 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:35.988841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d70037-e337-4d3c-bf5a-602173750c12 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:35.988841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d70037-e337-4d3c-bf5a-602173750c12 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:35.988841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 411/807] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:35.988841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d70037-e337-4d3c-bf5a-602173750c12 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:35.989398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07706039-ed6e-447f-bfd6-de67fe0adf66 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.095283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d70037-e337-4d3c-bf5a-602173750c12 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.132787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.133450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.133737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.134019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.134741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:36.135039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.135166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.135702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.135985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.136288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.136450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.137121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:36.137427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.138836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104f5aa8-d04c-489a-9f50-9cbe7a7df680 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.149465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 412/808] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:36.150266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.156607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 397/809] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Jul 15 19:12:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:36.157942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.158226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.158504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.175556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:36.177044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:36.180359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.243490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 398/810] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:36.244284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.252308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87363396-ccaa-441d-b975-eefdb14a1ca2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.270831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.286533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.287187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.287466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.287744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.342364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f072c701-793d-45ad-ad6c-39844d12ed9f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.354136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.355630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.376241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.377044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.377319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.377590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.378239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.380349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.384244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:36.392552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:36.393649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.394507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 399/811] 10.222.0.17 () {56 vars in 1129 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:36.395337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.458645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 413/812] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:12:36.459479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.489066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.498844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180abc2-1a31-4b33-8da2-ec67a74636b1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.534479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.535832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.536148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.536496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.543257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.544826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:36.559206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:36.560249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.561153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 400/813] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:36.561635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55ce0f-76fa-40eb-b9e4-26b8a211e353 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.561709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.586469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.594300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.595085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.595363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.595867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.605987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.607435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:36.625577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:36.626583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:36.627455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:36.627455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:36.627455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:36.627455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:36.627455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 414/814] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:36.627733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.669963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31555c7-b3b2-46b9-963d-1b7792c014f9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.703527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.704347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.704631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.704903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] PATH_INFO: `/v3/services/network` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.705547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.706909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:36.718662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:36.719674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.720529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 401/815] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:36.721333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.728243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c4c2-faa6-454d-8fd6-d40353ff2e7b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.743902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.763895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.764506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.764853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.765198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.765970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:36.766285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.767791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.773904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 415/816] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:36.774622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.822084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.828227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793d6fbe-17d7-44c6-b803-9395eeea1aa2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.861437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.862221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.862505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.862772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.874131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.875560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.877466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba33912-c5c3-48a9-896e-ae032d03f245 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:36.892825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:36.893722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:36.895357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:36.895357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:36.895357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:36.895357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:36.895357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 402/817] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:36.895357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.912070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:36.912754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:36.913034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:36.913306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:36.914081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:36.914385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:36.915811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:36.927658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 416/818] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:36.928463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:36.962057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:36.996485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023ac8d-a2d4-41f2-9fa0-3c36c764456a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.025624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac4720-0808-494d-8a42-1493d9cf5b11 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.027814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.030054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.030670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.030951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.031225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.031948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:37.032282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.033651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.045421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 403/819] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:37.046209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.063628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.064326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.064615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.064890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.065596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] RBAC: Adding query filter params (name=network) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:37.065900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.067849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.072380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 417/820] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:37.073363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.145814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c24955-9f14-4362-b287-592c983100b1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.170485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6588d-876b-4c32-b62d-a31b0fb01cdb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.172308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.180627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.181284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.181563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.181836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.182573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:37.182877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.184262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.188958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 404/821] 10.222.0.17 () {56 vars in 1124 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:37.189700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.208622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.209281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.209562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.209837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.210551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:37.210856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.212237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.223664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 418/822] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:37.224461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.251560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.289423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796b54c-2520-46d2-9cea-2f432f553a1d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.322656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.322902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52252658-c7b6-4c8a-b80c-a82db60b49d8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.323535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.323816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.324111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/62f4fa05f65c4643898254268882b8f2/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.338424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=62f4fa05f65c4643898254268882b8f2, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.339847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.364117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 405/823] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:37 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/62f4fa05f65c4643898254268882b8f2/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:37.364117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.368145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.369009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.369304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.369584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.377481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.379528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:37.396319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:37.397301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:37.398237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 419/824] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:37.398722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.408835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.476964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e6d38-d7f3-49ab-a243-d059d19cd27a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.501061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.519749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f84601-452d-423d-9e45-105dc95baf23 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.521664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.522342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.522630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.522911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.523625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] RBAC: Adding query filter params (type=network) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:37.523934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.525354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.530880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 406/825] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:37.531681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.560793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.561387 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.561670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.561944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.562677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.564047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.587617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 420/826] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:12:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:37.588671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.650834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7bff8-de3f-47ab-84a4-2e1f9ad24f27 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.661469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.700363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.701506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.701936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.702357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.710998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.712880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:37.729547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:37.730691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:37.731718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:37.739433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 407/827] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:37.745836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406fd2c-ea90-4257-94f0-09f684c7121e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.746372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.796930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.797605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.797941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.798223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.799017 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:37.799325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.801535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.818202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 421/828] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:37.818994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:37.825824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:37.884562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae420be-d96d-4225-9478-390433898bb4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.919683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:37.920375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:37.920655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:37.920925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:37.922114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:37.923136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:37.954346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 408/829] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:12:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:37.956131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22cfdc-7faf-4d03-80ba-3d03c1b49d76 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:37.962105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:38.003304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:38.045267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:38.046000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:38.046280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:38.046554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:38.049647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:38.049966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:38.052832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:38.076887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 422/830] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:38.083679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:38.170298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df048e-5b33-4976-8161-7aee06b80da5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:38.186729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:38.223727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e381e7-50ea-4fac-ace0-547c41e5d2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:38.225272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e381e7-50ea-4fac-ace0-547c41e5d2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:38.225568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e381e7-50ea-4fac-ace0-547c41e5d2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:38.225900 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e381e7-50ea-4fac-ace0-547c41e5d2f4 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/8b3c45809bac4828bcde30494ab7a0d2/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:38.228790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e381e7-50ea-4fac-ace0-547c41e5d2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=8b3c45809bac4828bcde30494ab7a0d2, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:38.230834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e381e7-50ea-4fac-ace0-547c41e5d2f4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:38.234095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbb4c8-0176-4390-855e-acd05c60907e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:38.249868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 409/831] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:37 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/8b3c45809bac4828bcde30494ab7a0d2/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:38.284925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:38.285627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:38.286022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:38.286309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:38.287160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=37f8217df17a44a989ce1c7ab8a9ac1a) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:38.287478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:38.288993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe7bf03-3bb2-4d01-83ef-a3524aa91ff6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:38.296322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 423/832] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:38 2022] GET /identity/v3/role_assignments?user.id=37f8217df17a44a989ce1c7ab8a9ac1a&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:39.475030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f04c2-5cff-40e0-8639-0a96fd0c98f0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:39.475546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f04c2-5cff-40e0-8639-0a96fd0c98f0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:39.475994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f04c2-5cff-40e0-8639-0a96fd0c98f0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:39.478249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 410/833] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:39.486426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56f3b3-a553-4b33-b309-399eea5de759 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:39.486946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56f3b3-a553-4b33-b309-399eea5de759 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:39.487384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c56f3b3-a553-4b33-b309-399eea5de759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:39.514923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6c56f3b3-a553-4b33-b309-399eea5de759 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:39.516943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c56f3b3-a553-4b33-b309-399eea5de759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:39.630571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 424/834] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:39.668417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f04c2-5cff-40e0-8639-0a96fd0c98f0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:39.713156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8daa-610d-4afc-84c4-960f55a7ff63 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:39.713464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8daa-610d-4afc-84c4-960f55a7ff63 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:39.714331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a8daa-610d-4afc-84c4-960f55a7ff63 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:39.717612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 425/835] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:39.722404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:39.722914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:39.723417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:39.754803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:39.757938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:39.789950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:39.815778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f04c2-5cff-40e0-8639-0a96fd0c98f0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:39.839948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:39.852627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:39.853597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:39.853972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:39.854318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:39.864589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 426/836] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:39.870048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:39.879467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:39.880403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:12:39.881907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:39.885895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 411/837] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:12:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:39.892320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:39.894224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:39.993533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bcde43-c0c6-447b-ae11-5b0da2f307aa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.028368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.029072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.029349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.029612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.030267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.031635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:40.044359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:40.046271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:40.047939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:40.051443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 427/838] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:40.051679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.089445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f232ba7-8880-4eae-b43f-b83f35eeca6e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.105861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:40.146932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.147960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.149001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.149487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.150763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:40.151520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.153890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.163113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:40.165251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed5fd6a-9802-4605-b6f3-fdeddb004a45 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.176420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 412/839] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:12:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:40.178903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.205553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.206218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.206498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.207305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.208222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:40.208531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.209909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.219378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 428/840] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:40.220607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.307400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e5044-e26d-4a88-83b7-754d4c446006 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.343409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e4ff6-6d52-4054-8a07-6301ecd747b1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.349537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.351112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.351638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.352153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.357124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:40.371112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.374136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.381529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.382228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.382507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.382781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.383492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:40.383794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.385575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.396610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 429/841] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7458ef19-177b-4eb0-9d0a-4fac006b66cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:40.398677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:40.399567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:40.400493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:40.400493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:40.400493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:40.400493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:40.407410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 413/842] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:40.408746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.485477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a942a-0b9a-4070-bfad-a743838bc429 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.486456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a942a-0b9a-4070-bfad-a743838bc429 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.487057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76a942a-0b9a-4070-bfad-a743838bc429 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.489852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 414/843] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:40.496901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.497512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.498260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.517963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:40.519674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:40.580197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:40.607115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 415/844] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:40.632300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.643937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7f71cf-f120-4c8f-9176-b7a5545a132b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.684603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.685781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.686265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.686745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.687790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:40.688341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.690051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601803eb-8ccf-440c-85a0-3e42a041e914 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.704229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 430/845] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:40.710044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559323c8-fb6e-4cd9-9bb8-3d90f232f7ac None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.710333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559323c8-fb6e-4cd9-9bb8-3d90f232f7ac None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.710613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559323c8-fb6e-4cd9-9bb8-3d90f232f7ac None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.712923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 431/846] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:40.712923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559323c8-fb6e-4cd9-9bb8-3d90f232f7ac None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.752988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5bfaa-f199-4024-b77f-ee7749d96ab6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.755560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:40.797551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9e2ec-bf1e-4a16-b7e6-130ce340cf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.798348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9e2ec-bf1e-4a16-b7e6-130ce340cf08 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.798633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9e2ec-bf1e-4a16-b7e6-130ce340cf08 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.798910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9e2ec-bf1e-4a16-b7e6-130ce340cf08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.810605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9e2ec-bf1e-4a16-b7e6-130ce340cf08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9eb9e2ec-bf1e-4a16-b7e6-130ce340cf08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:40.819188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:40.820157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:40.821003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 416/847] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:40.822153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.822153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.822317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.830100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559323c8-fb6e-4cd9-9bb8-3d90f232f7ac None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:40.839449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:40.840602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:40.870388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:40.871205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:40.871502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:40.871761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:40.879174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:40.880542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:40.882051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:40.898021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:40.899004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:40.899934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:40.903975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 432/848] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:40.904883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:40.917718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 417/849] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:40.918814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.013576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.022447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23775a51-69a8-4b7e-a6e1-52f8741606cf None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.038770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b20ae-c60b-4446-a6b9-605965803b77 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.056776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.057450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.057736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.058018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.058765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:41.059076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.060498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.070904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 433/850] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:41.071798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.076413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.077417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.078554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.078850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.079695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:41.080004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.081454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.101010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 418/851] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Jul 15 19:12:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:41.101769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.134650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.182796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e6d10-ad5c-4b3e-9a76-53a116b4b885 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.212156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c0974-a20a-4e8f-9471-176d0894ea72 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.218415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.219235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.219527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.219813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.220496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.221875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:41.234531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:41.235447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:41.236871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 434/852] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:41.237700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.253199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.254689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.255146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.255433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.262792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.265642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.267260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:41.286663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:41.287588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:41.288542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:41.288542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:41.288542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:41.288542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:41.288542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 419/853] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:41.288837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.351188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad199182-400e-4c51-8eb1-576ed6c08593 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.374775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.389048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.389704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.389991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.390298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.396780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfbe9d-92f8-4a99-a8ab-16f7a9b92176 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.434144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.435201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.435510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.435791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.436549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:41.437225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.439044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.444035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 420/854] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:41.445068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.470768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.472423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.499505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.548639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aca7a6-88ff-4f75-9929-b9c8fec54401 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.590018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 435/855] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:12:41.591160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.595645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.596562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.597015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.598021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.598917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:41.599374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.601537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.613572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 421/856] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:41.615088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.628460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.694996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36fc616-3d25-4c32-b1dd-92c15b8f65a0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.717468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.730391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.731048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.731324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.731596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.732328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:41.732453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc179f4-b08d-44cb-9450-fdf231adf8cf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.732625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.733993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.738550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 436/857] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:41.739327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.789972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.790926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.791210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.791485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.805888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.807375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:41.828939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:41.829902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:41.830790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:41.830790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:41.830790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:41.830790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:41.830790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 422/858] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:41.835824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.884147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c5e9e-ee73-4f69-abe3-68c52322c7b0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:41.924322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:41.930192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:41.930999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:41.931281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:41.931554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:41.941724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:41.943394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:41.964894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:41.966095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:41.967247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 437/859] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:41.974250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:41.980428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7478b36b-4e60-42c9-a15f-eb199852d8a8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.021493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.022175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.022458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.022996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.023738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:42.024045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.025700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.026546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.044951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 423/860] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:42.049706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.092842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d449f5a0-3925-449b-98a9-f8e3da73a2e0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.143168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.143885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.144708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.145719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.146856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:42.147175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.149282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.165759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7724c6-759a-4f97-ac39-5cc7a6373df2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.176655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 438/861] 10.222.0.17 () {56 vars in 1135 bytes} [Fri Jul 15 19:12:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:42.178854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.204901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.206085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.207097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.207819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.208910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.216596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.218348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:42.234671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:42.235599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:42.236533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:42.239302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 424/862] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:42.245362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.292944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86eb7-ca65-4db9-a1e9-15cca5d74145 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.296508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.335565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74adbcaf-23af-4b9d-8955-dc220415b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.336263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adbcaf-23af-4b9d-8955-dc220415b85c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.336551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adbcaf-23af-4b9d-8955-dc220415b85c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.336830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74adbcaf-23af-4b9d-8955-dc220415b85c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.382191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3750102-c151-4a09-b753-c4b9586fb420 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.425981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adbcaf-23af-4b9d-8955-dc220415b85c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.427377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74adbcaf-23af-4b9d-8955-dc220415b85c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.441420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.443359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.443654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.443935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.445051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:42.445367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.447580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.472280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 425/863] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:42.474345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.522207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.544282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 439/864] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:12:42.545610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44704cb-4ec5-422f-8228-97ed390b3e25 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.545904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44704cb-4ec5-422f-8228-97ed390b3e25 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.546184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44704cb-4ec5-422f-8228-97ed390b3e25 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.547631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 440/865] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:42.550234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44704cb-4ec5-422f-8228-97ed390b3e25 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.555696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.597169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62b408-0948-4804-b049-c72e5f0c6f1e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.641132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.641783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.642069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.642343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.643172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=62f4fa05f65c4643898254268882b8f2) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:42.643491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.644969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcc909-318e-4ae6-87b3-be43a3069ee8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.656117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 426/866] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/role_assignments?user.id=62f4fa05f65c4643898254268882b8f2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:42.656117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.656117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.656117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.666197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44704cb-4ec5-422f-8228-97ed390b3e25 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.683010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:42.684511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:42.713343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.714480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.714819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.715105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.730292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.732750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:42.757863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:42.759014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:42.759988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:42.759988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:42.759988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:42.759988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:42.759988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 441/867] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:42.764943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.789049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 427/868] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:42.808795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.836110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.847111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:42.893497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cab6960-e599-48b3-8b0d-63f89e7f30fe None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.945664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203d073-2cf0-4edf-8e91-2693c19f82a5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:42.946306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.946993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.947283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.947571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:42.948351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:42.948667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:42.950065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:42.963342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 442/869] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:42.969056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:42.997705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:42.998658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:42.999350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:42.999639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:43.000403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:43.002573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:43.021744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 428/870] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:12:42 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:43.089842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c5e8a-5dbb-4aba-aed3-a668325b0964 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:43.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:43.136729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:43.137553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:43.137841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:43.138120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:43.146424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:43.147931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:43.163736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:43.164903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:43.166174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 443/871] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:43.173492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:43.283292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:43.292927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa244a-31a9-4bf1-becb-0f56ad279c04 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:43.328768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:43.329438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:43.329721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:43.329997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:43.330792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:43.331100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:43.332665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f25f1eb-fa2c-42fb-b0e2-00cb458d5229 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:43.346390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 429/872] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:43.352974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:43.459774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1620048-60fa-4a1a-a8c8-51d40517b5a6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:43.495565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:43.496258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:43.496546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:43.496829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:43.497656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=8b3c45809bac4828bcde30494ab7a0d2) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:43.498283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:43.499732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26dfb9-021a-426f-8cd7-0b60f346eb7d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:43.506394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 444/873] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:43 2022] GET /identity/v3/role_assignments?user.id=8b3c45809bac4828bcde30494ab7a0d2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:43.929041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70f0e-e426-4080-81b3-9ab0647c2c6a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:43.929542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70f0e-e426-4080-81b3-9ab0647c2c6a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:43.929618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e70f0e-e426-4080-81b3-9ab0647c2c6a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:43.931152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 430/874] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:43.936476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a3c38-fbde-4b01-a396-b201f5682a61 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:43.936871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a3c38-fbde-4b01-a396-b201f5682a61 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:43.937040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6a3c38-fbde-4b01-a396-b201f5682a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:43.954221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fd6a3c38-fbde-4b01-a396-b201f5682a61 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:43.955411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fd6a3c38-fbde-4b01-a396-b201f5682a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:43.987103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:44.026829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 445/875] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:44.049137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70f0e-e426-4080-81b3-9ab0647c2c6a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:44.162825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e70f0e-e426-4080-81b3-9ab0647c2c6a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:44.198229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9b1e8-cb69-40fd-828f-49b8db96ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:44.198904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b1e8-cb69-40fd-828f-49b8db96ee6a None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:44.199190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b1e8-cb69-40fd-828f-49b8db96ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:44.199467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9b1e8-cb69-40fd-828f-49b8db96ee6a None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:44.200242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9b1e8-cb69-40fd-828f-49b8db96ee6a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:44.201617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d9b1e8-cb69-40fd-828f-49b8db96ee6a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:44.217290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 431/876] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:12:44 2022] POST /identity/v3/services => generated 269 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:44.901175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d97e-c318-4dcb-be56-262f63fd7e63 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:44.901871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d97e-c318-4dcb-be56-262f63fd7e63 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:44.902403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce9d97e-c318-4dcb-be56-262f63fd7e63 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:44.904234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 446/877] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:44.909139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ff7ee-0957-4fcb-b36b-17bc02467b59 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:44.910212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ff7ee-0957-4fcb-b36b-17bc02467b59 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:44.910834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ff7ee-0957-4fcb-b36b-17bc02467b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:44.928125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e58ff7ee-0957-4fcb-b36b-17bc02467b59 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:44.929635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e58ff7ee-0957-4fcb-b36b-17bc02467b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:44.938231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.011354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 432/878] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:45.033873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d97e-c318-4dcb-be56-262f63fd7e63 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.141059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce9d97e-c318-4dcb-be56-262f63fd7e63 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.177507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.178372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.178662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.178938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.189900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:45.197157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262abfb-908f-4ffb-97a0-1ec9210830cd None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.198069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262abfb-908f-4ffb-97a0-1ec9210830cd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:45.198462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:45.199349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:45.200550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 447/879] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:45.202494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2262abfb-908f-4ffb-97a0-1ec9210830cd None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.203979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 433/880] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:45.209212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.209808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.210098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.210379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.228239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:45.229778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:45.250989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.304973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 434/881] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:45.330648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.353835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9249da-0367-49ed-b393-ac35194b432d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.402337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.403262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.403658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.403942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.404739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:45.405050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:45.406432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:45.408587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.418989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 448/882] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:45.427391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.454502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f75fcb-ecf9-4f8d-ba72-9bbdf919746a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.493049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.494285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.494886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.495455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.496372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:45.498372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:45.511965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:45.513004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:45.513872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:45.517023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 435/883] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:45.524847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.536460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.551306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fb5f63-48d8-4f1c-839c-56eb60387762 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.587478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.588650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.589137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.589627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.601588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:45.603423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:45.621762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:45.622668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:45.623794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:45.623794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:45.623794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:45.623794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:45.630527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 449/884] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:45.632113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.650064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce20e5a-ab39-4628-b79d-ff4154b9dc55 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.684973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.690705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.691398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.691774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.692054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.692819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:45.693126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:45.694490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d34878-03b4-4fdc-aa3d-f3aafc917912 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:45.703707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 436/885] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:45.703707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6965e-592a-4810-8de9-8c4feadbcae2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.703707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6965e-592a-4810-8de9-8c4feadbcae2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.703707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6965e-592a-4810-8de9-8c4feadbcae2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.703707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 437/886] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:45.707007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6965e-592a-4810-8de9-8c4feadbcae2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.750538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d17901-b5bb-4de2-a4ef-9e13b339a4f0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.790904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.791574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.791858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.792158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.792885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:45.793195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:45.794569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957e2950-8f1a-42cf-81dd-dd8ccd68271a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:45.804291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.813034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 450/887] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:45.814276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.814566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.814844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.820190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6965e-592a-4810-8de9-8c4feadbcae2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:45.831714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:45.832971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:45.855261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:45.855931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:45.856248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:45.856527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:45.857235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:45.857541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:45.858903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:45.864905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 438/888] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:45.870186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.905191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 451/889] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:45.906613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:45.939899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:45.982533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac596dc-7ec1-4364-8d05-0a4f0a271114 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.017186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.017994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.018271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.018541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.024288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df036afe-004a-43bd-8e87-8da7e2bc05e7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.026740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.028205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:46.043438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:46.044454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:46.045384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 439/890] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:46.046428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.059938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.068448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.069351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.069817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.070281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.071160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] RBAC: Adding query filter params (interface=public, service_id=eee372a7af0043989faca5be95cdea60, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:46.071528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.073011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.082911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 452/891] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:12:45 2022] GET /identity/v3/endpoints?service_id=eee372a7af0043989faca5be95cdea60&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:46.082911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.157105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287f78-841e-4ad3-be2b-4d04eb0111e2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.179928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.193413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce9d80-aed9-4464-adec-cd50f3d2129d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.195795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.196747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.197027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.197299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.197954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.199327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:46.212304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:46.213493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:46.214298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:46.218417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 440/892] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:46.219269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.231473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.232186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.232472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.232746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.233582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:46.233888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.235314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.255115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 453/893] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:46.255724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.307778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.326423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b076c4-6a8c-41cf-8362-f932ff0e7aeb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.353646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef2269-d975-4e6d-9854-47cf1c7b0d8e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.359538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.360227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.360513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.360789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.361464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.362878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.368476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 441/894] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:12:46.369242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.390887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.391583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.391870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.392187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.392920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:46.393316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.395094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf094515-94a1-4903-a216-b4fa2bc7b9db None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.400499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 454/895] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:46.401853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020726d-5a4d-4046-a020-78b9127d2e41 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.402139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020726d-5a4d-4046-a020-78b9127d2e41 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.402419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020726d-5a4d-4046-a020-78b9127d2e41 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.408008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 455/896] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:46.408788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020726d-5a4d-4046-a020-78b9127d2e41 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.439260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.501143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038426f4-ea3f-465f-bf74-ee0a1110619e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.527392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020726d-5a4d-4046-a020-78b9127d2e41 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.539067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.540924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.540924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.540924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.546180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.566270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.567374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.567880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.568436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.569716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:46.570282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.571957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc43e8-39e5-4837-a974-a758ab26a4e9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.580353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 456/897] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:46.580353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.580353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.580353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.604863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:46.606202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:46.623201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.625038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.692620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 457/898] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:46.696481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.716055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.752321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 442/899] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:12:46.758529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.845806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebe6be-9819-4eb0-94d4-e86d035f0ba7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.858903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.885049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.886245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.886731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.887206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.887418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d80035-c621-402e-93c6-0b0c41a28ea1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:46.888324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.889919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:46.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:46.904582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:46.905403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:46.908394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 458/900] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:46.920106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:46.929027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:46.930222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:46.930716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:46.931549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:46.948288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:46.950622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:46.962756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:46.976686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:46.977599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:46.978486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:46.978486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:46.978486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:46.978486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:46.980167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 443/901] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:46.984301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.043762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af80a3-15a0-44a3-8745-0f786ba457ab None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.093083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08fe1e9-920f-4304-979c-53a9ec5182b0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.093515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.094196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.094482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.094761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.096687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:47.097060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.098467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.104764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 459/902] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:47.110615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.132611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.133653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.134192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.134667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.135663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:47.136204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.137790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.155456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 444/903] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:47.157130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.160617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:47.221063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:47.237192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0d285-e264-476b-9ff6-7cbb740f68df None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.271540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aab8ba-8cba-447b-967d-79b3e864cfa1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.275536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.276483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.276968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.277443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.278376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:47.278975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.280657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.291724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 460/904] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:47.293255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.310591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.311977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.312294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.312574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.321001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.323163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:47.343006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:47.343914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:47.344856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:47.348158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 445/905] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:47.350728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.409414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e0562-c6d8-4be0-8de0-2188aae4d07b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.411929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:47.452158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.453170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.453758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.454239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.455460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] RBAC: Adding query filter params (interface=public, service_id=c85eb9efd6474b149f32317a387487d9, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:47.455998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.458308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.468593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 461/906] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:12:47 2022] GET /identity/v3/endpoints?service_id=c85eb9efd6474b149f32317a387487d9&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:47.470022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.475687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9902152-36d2-4e43-b979-20c773bf3b14 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.501327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:47.519656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.520374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.520668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.520944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.521759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:47.522093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.524316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.544182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 446/907] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:47.546966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:47.593567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ace92-2d4e-43cf-b38f-40f5d48beec6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.631216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4490773c-10b3-4368-acef-bfa8c18dea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.631931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490773c-10b3-4368-acef-bfa8c18dea6c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.632343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490773c-10b3-4368-acef-bfa8c18dea6c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.632625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4490773c-10b3-4368-acef-bfa8c18dea6c None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.633315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490773c-10b3-4368-acef-bfa8c18dea6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.634781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4490773c-10b3-4368-acef-bfa8c18dea6c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.644635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 462/908] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:47 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:12:47.673830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a07608e-6ce0-43f3-b55b-a07e444bfdee None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:47.712999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:47.714154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:47.714651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:47.715127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:47.716214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] RBAC: Adding query filter params (role.id=cf194dc6f4f04e3ba6db84118c7a069a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=62f4fa05f65c4643898254268882b8f2) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:47.716742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:47.718433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3f4b6-fbdc-46d1-ad9b-0f77f433e632 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:47.726088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 447/909] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:47 2022] GET /identity/v3/role_assignments?user.id=62f4fa05f65c4643898254268882b8f2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=cf194dc6f4f04e3ba6db84118c7a069a => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:48.449557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af660346-e68f-4393-93aa-702f180ba4b6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.450108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af660346-e68f-4393-93aa-702f180ba4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.450108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af660346-e68f-4393-93aa-702f180ba4b6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.451857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 463/910] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:48.458024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d48337-ae2a-418d-be5b-037678bd8fe4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.458452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d48337-ae2a-418d-be5b-037678bd8fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.458670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d48337-ae2a-418d-be5b-037678bd8fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.476645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b9d48337-ae2a-418d-be5b-037678bd8fe4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:48.477831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b9d48337-ae2a-418d-be5b-037678bd8fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:48.521299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:48.554956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 448/911] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:48.573908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af660346-e68f-4393-93aa-702f180ba4b6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:48.605817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:48.656562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e17-ea0d-4e10-92bf-48ddb7400a40 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.656910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e17-ea0d-4e10-92bf-48ddb7400a40 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.657112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e17-ea0d-4e10-92bf-48ddb7400a40 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.658737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 449/912] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:48.664007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.664324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.664604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.683082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:48.684680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:48.686179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af660346-e68f-4393-93aa-702f180ba4b6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:48.723915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:48.724250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.724450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.724722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.725370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:48.726746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:48.742354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:48.743363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:48.744226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 464/913] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:48.754148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:48.776387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 450/914] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:48.794035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:48.865013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:48.873704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065840fd-a3b1-439e-83bd-708804fa07d8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:48.900875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97b10d-b1b2-4b5e-9d5b-5d554a102b1e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:48.903217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:48.915639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:48.916728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.917179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.917608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.918453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:48.919024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:48.920688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:48.927091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 465/915] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:48.937048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:48.937304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:48.937934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:48.938214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:48.938486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:48.939204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:48.940615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:48.964389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 451/916] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:12:48 2022] POST /identity/v3/services => generated 241 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:12:49.057753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c19580-4510-446f-9937-112c7f40ce99 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:49.101388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:49.102064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:49.102347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:49.102622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:49.103325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:49.103628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:49.105023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b9ddf6-7e40-48ea-919b-0fd40225b5d4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:49.110543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 466/917] 10.222.0.17 () {56 vars in 1114 bytes} [Fri Jul 15 19:12:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:49.116625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:49.226473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b8093d-836c-4aa9-9ed9-47b6be996bf8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:49.264540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225d8e5-1411-4e02-9808-d17f33575f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:49.265211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225d8e5-1411-4e02-9808-d17f33575f9d None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:49.265489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225d8e5-1411-4e02-9808-d17f33575f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:49.265761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225d8e5-1411-4e02-9808-d17f33575f9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:49.266506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225d8e5-1411-4e02-9808-d17f33575f9d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:49.267875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225d8e5-1411-4e02-9808-d17f33575f9d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:49.291221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 452/918] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:12:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:49.758115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b7bb-a78f-4fff-bc2e-7347778ffbbe None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:49.758738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b7bb-a78f-4fff-bc2e-7347778ffbbe None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:49.758738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376b7bb-a78f-4fff-bc2e-7347778ffbbe None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:49.760520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 467/919] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:49.765819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:49.766257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:49.766412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:49.785861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b86c73-b5b1-4afb-a8bd-505cb231466d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:49.786187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b86c73-b5b1-4afb-a8bd-505cb231466d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:49.786424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b86c73-b5b1-4afb-a8bd-505cb231466d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:49.788121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 468/920] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:49.792275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:49.793255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:49.793543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:49.793822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:49.794260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:49.810283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:49.814911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:49.816524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:49.833628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:49.878006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 453/921] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:49.898020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 469/922] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:49.908102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:49.919566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.012819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059cb9e-236c-4b90-81eb-07627f446ba4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.038792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2917-a04c-44e7-94a2-cf7993f4fa8c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.049392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.050785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.051221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.051494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.052179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.053557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:50.066181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:50.067437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 15 19:12:50.068746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:50.072251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 454/923] 10.222.0.17 () {56 vars in 1132 bytes} [Fri Jul 15 19:12:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:50.073157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.082792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.083793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.084337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.084746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.085906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:50.098621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:50.107666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:50.108647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:12:50.109484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:50.112559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 470/924] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Jul 15 19:12:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:50.116053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.130743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:50.197314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fea73-168d-4beb-97c4-fbfd5e67435d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.234177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572bdea-a1ec-43ee-ac1d-524b31646af9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.244823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.246060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.246551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.247274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.248671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:50.248981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.250360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.263470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 455/925] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:50.265586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.280217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.280870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.281151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.281421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.282162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:50.282461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.283878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.292617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 471/926] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:50.297673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.370402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:50.389273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:50.404657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce7bc66-c243-4ac3-84dd-b24c63731d74 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.430834 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef0d512-e4f6-4dfa-8bf4-3a74ae71304b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.449289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.450087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.450467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.450837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.451641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:50.452566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.454355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.462636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 456/927] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:50.468290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.475182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.476056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.476436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.476774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.492250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.493924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:50.521598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:50.522732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:50.523676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:50.523676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:50.523676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:50.523676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:50.530164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 472/928] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:50.531205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.603103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11287140-5079-404d-bd4f-f84d72178c2e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.647256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a30ce1-dbfa-4009-b68d-135511a296e6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.653538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.654478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.655558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.656034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.657551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.659602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.669949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:50.673079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:50.696365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 457/929] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:12:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:12:50.702634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.703371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.703660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.704002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.704835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:50.705151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.706680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.721376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 473/930] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:50.727782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:50.855763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621268bb-c1cf-4812-836b-9f9ab1e16e20 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:50.894010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:50.895180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:50.895685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:50.896175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:50.904293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:50.905951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c42b7c44-eba8-4c9d-b894-c6a1fa073a46 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:50.921449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:50.922461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:50.923443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 458/931] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:50.934141 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.043642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ea30e-7591-451a-8b8c-811ae4a294fc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.079940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.080640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.080923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.081748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.082595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:51.082900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.084349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4aa5d4-b7aa-4f5b-96d3-c9d6931ba9e6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.101803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3420-1350-4fa4-a7d1-ce2ed887ba94 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.102120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3420-1350-4fa4-a7d1-ce2ed887ba94 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.102352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af3420-1350-4fa4-a7d1-ce2ed887ba94 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.103251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 474/932] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Jul 15 19:12:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:51.103928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 459/933] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:51.108470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af3420-1350-4fa4-a7d1-ce2ed887ba94 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.109011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.109304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.109615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.126920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:51.128244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:51.161275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.177541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.212350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 475/934] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:51.229637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.229927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af3420-1350-4fa4-a7d1-ce2ed887ba94 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.271158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.272158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.272567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.272947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.288814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.290968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:51.322717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:51.323777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:51.324705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:51.324705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:51.324705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:51.324705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.325062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 460/935] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:51.330033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.345121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920dcd8-d5b6-48ae-83fe-a6635c300bb4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.381856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.383014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.383517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.384000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.384872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.386464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:51.399641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:51.400703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:51.401530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:51.408933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 476/936] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:51.411084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.441066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e1275-be72-4d12-9265-07ce6340a06c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.444341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.448739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.481260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.482216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.482499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.482774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.483498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:51.483803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.485202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4446a946-9224-478a-9ca8-97c5199187ac None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.499297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 461/937] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:51.500658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18e51b-f923-4934-af3a-0cadf32614f1 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.500945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18e51b-f923-4934-af3a-0cadf32614f1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.501215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe18e51b-f923-4934-af3a-0cadf32614f1 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.504124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 462/938] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:51.506358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18e51b-f923-4934-af3a-0cadf32614f1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.526832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6428a98-72de-404b-99b0-7d6cce1d555a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.561776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.562441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.562725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.563000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.563711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:51.564023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.565441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8165ec-efba-4afa-b1e4-61f145012003 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.576047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 477/939] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:51.576047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.576047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.576047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.591232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:51.592463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:51.632005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe18e51b-f923-4934-af3a-0cadf32614f1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.676105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 478/940] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:51.676105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.680583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.682066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.682515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.682859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.691195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.694220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.695714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:51.710601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:51.711501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.712416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 463/941] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:51.712918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.749924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.779954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aae7d6-5a89-470a-9dc5-5e5fe081506b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.816351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.817019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.817305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.817581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.818289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:51.819013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.820705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.832835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 479/942] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:51.832835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.833077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a93794-c2c2-4988-a9c9-ae710be97d9b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:51.865994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:51.866768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:51.867054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:51.867338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:51.868043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:51.869522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:51.870894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING keystone.server.flask.common [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 15 19:12:51.890301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.sql.core [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 19:12:51.890301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 19:12:51.890301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [parameters: {'id': '0fa651c365d94a5b8557fef5bdc88e9c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 19:12:51.890301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80200) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fa651c365d94a5b8557fef5bdc88e9c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fa651c365d94a5b8557fef5bdc88e9c. Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:12:51.898339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result.read() Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 15 19:12:51.899371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application yield resource Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise self.value Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application session.commit() Jul 15 19:12:51.900644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.session.flush() Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application compat.raise_( Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 19:12:51.901775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 19:12:51.903214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result.read() Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:12:51.904331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [parameters: {'id': '0fa651c365d94a5b8557fef5bdc88e9c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.905584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:51.906901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fa651c365d94a5b8557fef5bdc88e9c. Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 464/943] 10.222.0.17 () {60 vars in 1140 bytes} [Fri Jul 15 19:12:51 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:51.908113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:51.951661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:51.966009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad9a60-c529-44d4-994b-58542dbf4ca6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:52.002912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:52.005985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:52.006666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.006963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.007223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.008018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:52.008372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:52.009834 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:52.023717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 480/944] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:52.024725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7270842-3cd6-40df-93c1-70d3042fa8cb None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:52.025189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:52.063152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:52.063815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.064118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.064398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.065187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] RBAC: Adding query filter params (interface=public, service_id=09385087340546248fb71eb9dae30643, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:52.065503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:52.068309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:52.077522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 465/945] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:12:51 2022] GET /identity/v3/endpoints?service_id=09385087340546248fb71eb9dae30643&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:52.078301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:52.137149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a81ee-6e39-4de1-b8ad-e1c09d0eccb7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:52.173006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:52.173796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.174092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.174345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.181683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:52.183129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:52.188236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5362a8-6705-4cd6-922e-1596b66e8b6d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:52.197877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:52.199062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:52.199980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 481/946] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:52.200502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:52.214318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:52.235117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:52.236294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.236579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.236853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/62f4fa05f65c4643898254268882b8f2/roles/cf194dc6f4f04e3ba6db84118c7a069a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.240314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=62f4fa05f65c4643898254268882b8f2, role_id=cf194dc6f4f04e3ba6db84118c7a069a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:52.241740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:52.256164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 466/947] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:12:52 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/62f4fa05f65c4643898254268882b8f2/roles/cf194dc6f4f04e3ba6db84118c7a069a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:12:52.269715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:52.286186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:52.325011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84b7863-9a1d-4f6b-ac81-8d8d7fe0d1f7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:52.368513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672367c-3be3-477c-9c7b-2a4b8b55273a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:52.369373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672367c-3be3-477c-9c7b-2a4b8b55273a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.369761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672367c-3be3-477c-9c7b-2a4b8b55273a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.370271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672367c-3be3-477c-9c7b-2a4b8b55273a None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.371145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672367c-3be3-477c-9c7b-2a4b8b55273a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:52.373100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672367c-3be3-477c-9c7b-2a4b8b55273a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:52.388825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 482/948] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:52 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:12:52.423699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb77d7f-37f9-4a81-a0f9-2e2d67bc6941 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:52.463340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:52.464022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.464335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.464612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.465389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:52.465696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:52.467167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d823-d36d-4163-aec2-263d9452dd0e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:52.487234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 467/949] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:52.870140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36a029-5f42-4808-a396-74c3918d8861 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.870475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36a029-5f42-4808-a396-74c3918d8861 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.870694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36a029-5f42-4808-a396-74c3918d8861 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.872345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 483/950] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:52.878143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:52.878615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:52.878713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:52.896310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:52.897815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:52.959953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:52.972421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 468/951] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:52.990508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36a029-5f42-4808-a396-74c3918d8861 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:53.066752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:53.098268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36a029-5f42-4808-a396-74c3918d8861 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:53.136434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:53.137322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:53.137674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:53.138016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:53.138732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:53.140192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:53.151947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:53.153257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:12:53.154303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:53.155992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 484/952] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:53.162301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:53.265565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ef178-b9d7-4760-898a-5c86b6af4b1b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:53.302096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:53.302775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:53.303059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:53.303337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:53.304058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:53.304391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:53.305771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c08905-8eb4-48cc-a69f-f99199ef28f0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:53.311231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 469/953] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:53.317033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:53.423424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd90e8-d224-4cfb-a87b-02686ab58ce3 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:53.458490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:53.459153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:53.459487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:53.459766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:53.460549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:53.460859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:53.462225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762e2ce-3503-40e3-87c2-e9fc24ddf898 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:53.464351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:53.467239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 485/954] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:54.406209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40f762-dbb7-4430-97e1-aa41b1e1b1e0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.406764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40f762-dbb7-4430-97e1-aa41b1e1b1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.406764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40f762-dbb7-4430-97e1-aa41b1e1b1e0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.408748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 470/955] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:54.415018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.415473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.415734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.431704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dbdea-bbad-426d-ba98-484cb484d062 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.432039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dbdea-bbad-426d-ba98-484cb484d062 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.432301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3dbdea-bbad-426d-ba98-484cb484d062 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.433947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 471/956] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:54.436265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:54.437674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:54.442030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.442390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.442733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.463923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:54.466052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:54.536247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 486/957] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:54.559353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:54.563700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:54.576578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 472/958] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:54.599994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:54.675728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd346c-ca6c-4159-89f8-dbd352ebced1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:54.706152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:54.712475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:54.713289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.713575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.713849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.726219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:12:54.728875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bf513-02eb-4126-8579-7a6a521414fc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:12:54.734709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:54.735663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:12:54.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:54.740624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 487/959] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Jul 15 19:12:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:54.742488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:54.777175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:54.777963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.778249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.778532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.779192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:54.780608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-2f2f9fcb-6a53-4e23-ae86-1a0c35d18392 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:54.792746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:54.793767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:54.794603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 473/960] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:54.795413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77276fe-c401-40a5-b6f9-02cd10462b7c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.795481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77276fe-c401-40a5-b6f9-02cd10462b7c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.795897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77276fe-c401-40a5-b6f9-02cd10462b7c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.797159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 474/961] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:54.803370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.803669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.803938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.821989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:54.823175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:54.861710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b532126-71cb-4d1b-91f5-0ebc5da088ef None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:54.877072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:54.895761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:54.896482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:54.896767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:54.904423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:54.904536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 475/962] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:54.904618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:54.904690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:54.904761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:54.904761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:54.914459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 488/963] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:12:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:54.920335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:54.976677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.015594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5318db-8839-47d9-b7f7-6f5cc25880aa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.025444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16118d3e-f05d-41c4-8cee-670165d5ad46 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.053828 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.054615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.054921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.055176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.055881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:55.056233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.057794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.060040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.060854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.061139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.061415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.063532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 476/964] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:55.064879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.074990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.076910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:55.098414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:55.099546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:55.100676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:55.100676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:55.100676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:55.100676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:55.104236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 489/965] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:55.105228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.122902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.182919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff53f3-d34c-44d7-85e5-32af78c3f567 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.212496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6d983-b884-4bf9-9c64-c0a800531628 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.214915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.222700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.223730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.224018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.224328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.225353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.226839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.239854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.sql.core [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 19:12:55.239854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 19:12:55.239854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [parameters: {'user_id': 'f0b54b75251c410b907a92667043bb43', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 19:12:55.239854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80200) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result.read() Jul 15 19:12:55.247034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:12:55.248268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application yield resource Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application next(self.gen) Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.session.rollback() Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.force_reraise() Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise self.value Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application session.commit() Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 15 19:12:55.249279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._prepare_impl() Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.session.flush() Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._flush(objects) Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application compat.raise_( Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application flush_context.execute() Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rec.execute(self) Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 15 19:12:55.250162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 15 19:12:55.251065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result = self._query(query) Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application conn.query(q) Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application result.read() Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 15 19:12:55.251920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f0b54b75251c410b907a92667043bb43', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:55.255779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 15 19:12:55.258062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 477/966] 10.222.0.17 () {60 vars in 1131 bytes} [Fri Jul 15 19:12:55 2022] POST /identity/v3/users => generated 160 bytes in 190 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:55.261474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.263303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.263303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.263303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.263303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.263943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:55.263943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.263943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.271079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 490/967] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:55.271079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.364651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d7853-5606-4e5c-9419-b2086db3075b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.375486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.377466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549d4a-5553-42ed-b110-e2732c58ac59 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.402444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.403112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.403397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.403674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.404435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:55.404748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.406326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.416575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.417935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.418428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.418886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.421735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 478/968] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:55.422578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.432347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.433814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:55.450256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:55.451402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 491/969] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:55.452389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.483573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.540696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c87e782-a9e2-4303-8728-387f2699e413 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.556981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcffae-d99f-4ae4-acc4-798c723cea3b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.574272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.574927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.575217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.575504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.576286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.577658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.592008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.593106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.593557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.593878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.599704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 479/970] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:12:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:55.600543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.606235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.607726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-a0bb4302-d25c-4241-ae2d-d11185326450 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:12:55.624192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:55.625216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:55.626755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 492/971] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:55.627375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cb1cf-2340-4fba-b17d-7d0980720876 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.627375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cb1cf-2340-4fba-b17d-7d0980720876 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.627736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755cb1cf-2340-4fba-b17d-7d0980720876 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.633270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 493/972] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:55.633458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cb1cf-2340-4fba-b17d-7d0980720876 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.648157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.745146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b04f74-78a9-4ef5-83f0-7056ee0d7daf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.747426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.764824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755cb1cf-2340-4fba-b17d-7d0980720876 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:55.785286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.785968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.786251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.786524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.787331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:55.787637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.789232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.803170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:55.803842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.804194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.804474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.805318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:55.805621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:55.807043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c95ed6-a583-49ad-a0f4-bce95d5c5bf5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:55.829536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 494/973] 10.222.0.17 () {56 vars in 1149 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:55.829536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:55.829536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:55.829536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:55.834149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 480/974] 10.222.0.17 () {56 vars in 1101 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:55.835074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:55.845562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:55.846799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:55.967439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 495/975] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:55.981907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:55.992802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:56.010555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89fba8-d7ad-485e-92a8-b9bfcb873e4d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:56.049185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:56.050937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:56.051423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:56.051940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:56.094997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:56.124246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1367f9-4ce4-406f-9470-71828d85b8e7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:56.149884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:56.151342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:56.163179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:56.163943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:56.164341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:56.164690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:56.165501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:56.166949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:56.184119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 496/976] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:12:55 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:12:56.247894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:56.275257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 481/977] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:12:55 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:12:56.276748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:56.392529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63ed659-3cb5-4a2c-b9fa-b14d75b4f629 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:56.422984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:56.429563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:56.430359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:56.430687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:56.431001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:56.442849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:56.444341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:12:56.462092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:56.463006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:56.463891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:12:56.463891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:12:56.463891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:12:56.463891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:56.463891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 497/978] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:56.468956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:56.571376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486135b8-4af4-4cda-815f-b69f20229cbd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:56.606124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:56.606787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:56.607070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:56.607347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:56.608068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:56.608402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:56.609767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:56.625920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 482/979] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:56.627114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:56.729922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c7ff9-f08e-4942-8de5-550e7bc473e1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:56.767827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:56.772071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:56.772071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:56.772071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:56.780952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:56.782414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:12:56.797658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:56.798619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:56.804501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 498/980] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:56.804993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:56.849249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:56.909744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8475cae-4651-4308-89ad-eac3ac09360a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:56.944065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:56.944770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:56.945055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:56.945334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:56.946117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:56.946429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:56.947855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3741c0-943a-4ed3-8526-8587b7374a76 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:56.966164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 483/981] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:56.967388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:57.013471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:57.070574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58701d26-757d-425c-828c-9af8b90298fb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:57.105468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:57.106122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:57.106407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:57.106687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:57.107516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] RBAC: Adding query filter params (role.id=cf194dc6f4f04e3ba6db84118c7a069a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=62f4fa05f65c4643898254268882b8f2) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:57.107828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:57.109302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d86519-a34f-4a5e-8e54-d34af82bfce7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:57.115381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 499/982] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:12:56 2022] GET /identity/v3/role_assignments?user.id=62f4fa05f65c4643898254268882b8f2&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=cf194dc6f4f04e3ba6db84118c7a069a => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:57.710378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab2b64-368a-467d-ac2a-b77468b6c734 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:57.710936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab2b64-368a-467d-ac2a-b77468b6c734 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:57.710936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab2b64-368a-467d-ac2a-b77468b6c734 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:57.712323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 484/983] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:57.717519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:57.717922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:57.718077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:57.735445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:57.736650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:57.812395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 500/984] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:12:57.834581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab2b64-368a-467d-ac2a-b77468b6c734 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:57.935187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:57.940061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab2b64-368a-467d-ac2a-b77468b6c734 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:57.974680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:57.975471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:57.975760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:57.976035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:57.976716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:57.978078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:57.990692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:57.991681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:12:57.992565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:57.996400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 485/985] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:12:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:57.998249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.039258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:58.102136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e474a6e3-25f9-4b68-a91c-0a25f17c3002 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.136574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.137239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.137521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.137797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.138510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:58.138817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.140229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9ca00a-5d84-4704-9545-f832a2929a9d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.145740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 501/986] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:58.151289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.241469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc3a4-f559-41f2-a372-bdc2757c5c12 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.241664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc3a4-f559-41f2-a372-bdc2757c5c12 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.241926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fc3a4-f559-41f2-a372-bdc2757c5c12 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.243327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 502/987] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:12:58.249039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.249368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.249602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.257936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c0b7d-ee12-419b-812d-deb69aaab6be None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.268202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:58.270693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:58.298915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.299619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.299897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.300484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.300901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:58.301208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.302567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.307830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 486/988] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:58.313509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.344144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 503/989] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:58.344627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:58.365744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.399096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:58.430878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea5838d-6baa-40ce-9304-6efc2575cb97 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.466471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.467130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.467415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.467689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.468513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] RBAC: Adding query filter params (interface=admin, service_id=09385087340546248fb71eb9dae30643, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:58.468823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.470178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.472476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae51d73-bfcc-4882-9ad0-cb689012847c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.475834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 487/990] 10.222.0.17 () {56 vars in 1249 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/endpoints?service_id=09385087340546248fb71eb9dae30643&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:58.481186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.508196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.508922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.509206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.509483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.510137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.511516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:58.523245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:58.524242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:12:58.525179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 504/991] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:12:58.530257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.589124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:58.600618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8ff7f-4b7a-48dd-be00-d1b59d46a2c8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.638056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.638727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.639027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.639280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.639959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.640960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:58.641367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.649044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c138be26-2c4e-48eb-95e3-3452df817538 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.649494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 488/992] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:12:58.700294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.700983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.701261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.701534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.702252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:58.702556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.704004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.714919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 505/993] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:58.716643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.835691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133abfee-dc69-4ad0-bc2e-90255d92f014 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:58.871043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:58.871716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:58.872003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:58.872309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:58.873285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:58.873586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:58.874957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:58.880926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 489/994] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:58.886609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:58.988968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244d621-1d72-4d03-876f-cf74071e68e1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:59.026803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:59.027389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.027674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.027953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.028781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] RBAC: Adding query filter params (interface=public, service_id=db1d4bdfea4246fe80139aa3702e4391, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:59.029094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:59.030470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd6a4f2-3df7-434c-8642-6c39e297ffda None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:59.040565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 506/995] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:12:58 2022] GET /identity/v3/endpoints?service_id=db1d4bdfea4246fe80139aa3702e4391&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:12:59.042142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:59.065968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:59.147201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde3072-2971-43d1-bfbe-842cbd724fe1 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:59.181123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:59.181795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.182075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.182317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06b47-48ff-4920-89de-25e013cbc7c4 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.182423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.182614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06b47-48ff-4920-89de-25e013cbc7c4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.182892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f06b47-48ff-4920-89de-25e013cbc7c4 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.182989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:59.184479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 507/996] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:12:59.184583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:59.189059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.189348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.189628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.200491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 490/997] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:12:59 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:12:59.207360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:12:59.208625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:12:59.229415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:59.287414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 508/998] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:12:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:12:59.311288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:59.403730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:12:59.421749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba1f33-e6b7-4bc2-913d-dce065e3db9b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:59.456845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:59.457708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.457991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.458273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] PATH_INFO: `/v3/services/image` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.458927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:59.460322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:12:59.472943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:12:59.473946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:12:59.474799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 491/999] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Jul 15 19:12:59 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:12:59.481797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:59.583336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793196b6-1d2b-49e3-8e92-d33d60b58570 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:59.618435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:59.619103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.619387 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.619664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.620411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] RBAC: Adding query filter params (name=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:59.620718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:59.622096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0095631d-9831-4e43-b120-27d0fefd2a22 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:59.627336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 509/1000] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:12:59.633200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:12:59.741131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94d2f0-5a14-4ab4-b7fb-6848da7ccce2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:12:59.781110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:12:59.781784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:12:59.782066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:12:59.782339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:12:59.783112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] RBAC: Adding query filter params (type=image) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:12:59.783417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:12:59.785372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ea3e3-abb0-4aa3-b51e-13d3a8a1592b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:12:59.796186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 492/1001] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:12:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:00.687009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec94f9-43d0-4231-bf77-4197c97a7681 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:00.687549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec94f9-43d0-4231-bf77-4197c97a7681 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:00.687549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec94f9-43d0-4231-bf77-4197c97a7681 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:00.688995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 510/1002] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:00.695344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:00.695641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:00.695921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:00.713525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:00.715048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:00.724209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:00.795361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 493/1003] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:00.819029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec94f9-43d0-4231-bf77-4197c97a7681 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:00.842630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:00.928443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec94f9-43d0-4231-bf77-4197c97a7681 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:00.964277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:00.965000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:00.965283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:00.965555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:00.966202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:00.967573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:00.979843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:00.980987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 15 19:13:00.981932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:00.984154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 511/1004] 10.222.0.17 () {56 vars in 1126 bytes} [Fri Jul 15 19:13:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:00.987497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.088198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a10f2-e030-4956-874c-70f17216dd1e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:01.121867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:01.122790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.123087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.123364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.124070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:01.124405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:01.125764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660c030-078b-4fbb-97bd-98b9fd7503ca None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:01.135313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 494/1005] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:13:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:01.136804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.219612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:01.240224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12edd1-8d9f-42f5-8eb6-888be2d5b0dd None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:01.251108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123feef3-86db-45ca-a964-4923826bcf97 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.251369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123feef3-86db-45ca-a964-4923826bcf97 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.251675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123feef3-86db-45ca-a964-4923826bcf97 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.256255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 495/1006] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:01.258354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.258622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.258834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.269719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:01.276925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:01.278576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:01.278687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:01.279361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.279647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.279935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.280679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:01.280988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:01.282371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:01.287995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 512/1007] 10.222.0.17 () {56 vars in 1122 bytes} [Fri Jul 15 19:13:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:01.293612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.354266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 496/1008] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:01 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:13:01.376943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.409529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a674a9-237e-4efa-94b2-d262d213b314 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:01.446498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:01.447159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.447800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.447800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.448560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:01.449943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:01.465286 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 513/1009] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:13:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:13:01.496032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b4767-2aa7-4762-9d05-8cb6718572b9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:01.536972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:01.543603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:01.544451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.544737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.545019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.545681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:01.547082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:01.560973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:01.562463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 15 19:13:01.563751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:01.567786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 497/1010] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:13:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:01.569138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.579192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:01.682440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f87ca-7084-4ff6-a531-54b57854b417 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:01.717251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:01.718034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.718242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.718519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.719410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:01.719718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:01.721126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c1d8e-82b6-441b-a894-815505fc0a73 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:01.731733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 514/1011] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:13:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:01.733311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.809535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30bac8-5358-455e-820b-be4b932ec262 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.809855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30bac8-5358-455e-820b-be4b932ec262 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.810094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb30bac8-5358-455e-820b-be4b932ec262 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.811610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 515/1012] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:01.816569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.816850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.817130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.834932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:01.836134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:01.844495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f29db4-8e9e-4e70-b33b-03fe850eef12 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:01.881681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:01.882346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:01.882625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:01.882902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:01.883609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:01.883914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:01.885326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:01.895326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 498/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Fri Jul 15 19:13:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:01.897130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.905736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:01.913102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 516/1014] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:13:01.936005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:01.961767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:02.006930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04cd632-517b-47ce-a1ad-e1ac3841d5b3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:02.042073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d95a279-7a7f-4a5d-8893-3c2ed11fb6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:02.042773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95a279-7a7f-4a5d-8893-3c2ed11fb6c2 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:02.043060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95a279-7a7f-4a5d-8893-3c2ed11fb6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:02.043346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d95a279-7a7f-4a5d-8893-3c2ed11fb6c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:02.044124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95a279-7a7f-4a5d-8893-3c2ed11fb6c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:02.045498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d95a279-7a7f-4a5d-8893-3c2ed11fb6c2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:02.046535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df86613-ea87-48df-b76e-61f66c36da3d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:02.066960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 499/1015] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:13:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:13:02.101268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a84ad-6423-46c6-b04c-a33e5eb33ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:02.101946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a84ad-6423-46c6-b04c-a33e5eb33ded None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:02.102228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a84ad-6423-46c6-b04c-a33e5eb33ded None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:02.102502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a84ad-6423-46c6-b04c-a33e5eb33ded None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:02.103227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a84ad-6423-46c6-b04c-a33e5eb33ded None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:02.104680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a84ad-6423-46c6-b04c-a33e5eb33ded None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:02.121076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 517/1016] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:13:01 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:13:03.534303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490687-30d5-4c85-abd8-755fd716ba7a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:03.534822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490687-30d5-4c85-abd8-755fd716ba7a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:03.534901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05490687-30d5-4c85-abd8-755fd716ba7a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:03.536323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 500/1017] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:03.541375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea863-66d6-4777-be26-a81345909823 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:03.541767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea863-66d6-4777-be26-a81345909823 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:03.541937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ea863-66d6-4777-be26-a81345909823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:03.558686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f63ea863-66d6-4777-be26-a81345909823 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:03.559873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f63ea863-66d6-4777-be26-a81345909823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:03.611060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:03.646019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 518/1018] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:03.668903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490687-30d5-4c85-abd8-755fd716ba7a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:03.777116 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05490687-30d5-4c85-abd8-755fd716ba7a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:03.794427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:03.814347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc829c-5c57-4c66-9886-ecce36928d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:03.815675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc829c-5c57-4c66-9886-ecce36928d03 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:03.815995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc829c-5c57-4c66-9886-ecce36928d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:03.816521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc829c-5c57-4c66-9886-ecce36928d03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:03.817777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc829c-5c57-4c66-9886-ecce36928d03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:03.819390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc829c-5c57-4c66-9886-ecce36928d03 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:03.826093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 501/1019] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:03.827010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea863-66d6-4777-be26-a81345909823 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:03.926747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ea863-66d6-4777-be26-a81345909823 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:03.960977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9cd53f-ee85-4a06-aeb8-0f3830a2696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:03.961638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd53f-ee85-4a06-aeb8-0f3830a2696a None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:03.961925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd53f-ee85-4a06-aeb8-0f3830a2696a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:03.962201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9cd53f-ee85-4a06-aeb8-0f3830a2696a None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:03.962909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9cd53f-ee85-4a06-aeb8-0f3830a2696a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:03.964381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9cd53f-ee85-4a06-aeb8-0f3830a2696a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:03.991351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 519/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:13:03 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:13:04.216241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccaa01-9cba-4f89-a9e9-e5226ce31d77 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.216534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccaa01-9cba-4f89-a9e9-e5226ce31d77 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.216812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cccaa01-9cba-4f89-a9e9-e5226ce31d77 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.218072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9640c-4a46-4709-b4fa-e546a667e2bc None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.218394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9640c-4a46-4709-b4fa-e546a667e2bc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.218504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 502/1021] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:04.218671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9640c-4a46-4709-b4fa-e546a667e2bc None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.220158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 520/1022] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:04.223756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.224045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.224347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.225938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117253a-07db-4538-b551-19de371393e4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.226228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117253a-07db-4538-b551-19de371393e4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.226508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117253a-07db-4538-b551-19de371393e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.241445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:04.242625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:04.243231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e117253a-07db-4538-b551-19de371393e4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:04.244595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e117253a-07db-4538-b551-19de371393e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:04.273942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:04.315853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 503/1023] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:04.325003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 521/1024] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:13:04.338733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.350005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117253a-07db-4538-b551-19de371393e4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.452391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842d72-4941-4f75-8896-73d1f188cdd1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.455308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117253a-07db-4538-b551-19de371393e4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.468453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:04.491277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:04.492076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.492385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.492662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] PATH_INFO: `/v3/services/image` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.493320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:04.493694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:04.494682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.494799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:04.495385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.495809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.507680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:04.507843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:04.508797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:04.514580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 522/1025] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:04.515370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:04.516374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:04.517308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:13:04.519109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:04.522803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 504/1026] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:04.523605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.548922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:04.628009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d70ff95-20cd-492d-aa7a-171101f1847d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.651784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5baf0-9471-404a-b2c1-c9ae754ee580 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.663399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:04.664067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.664376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.664653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.665385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:04.665690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:04.667043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:04.672127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 523/1027] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:04.678916 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.691264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:04.691959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.692277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.692554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.693304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:04.693601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:04.694962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:04.703229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 505/1028] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:04.708781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.746877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:04.798603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:04.810228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c8d142-b6c8-411b-8ac6-dc75bb88f2f2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.817339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85646f-8c4a-413a-9db0-17448b89c0bf None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.844469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:04.845142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.845442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.845711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.847161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] RBAC: Adding query filter params (type=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:04.847564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:04.849076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:04.854678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 524/1029] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:04.860440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.862297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:04.863379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:04.863832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:04.864282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:04.876707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:04.879587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:04.897053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:04.897958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:04.898846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:04.898846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:04.898846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:04.898846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:04.898846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 506/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:04.904062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:04.968408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29cf388-1901-4954-a1d2-b3807a776fcc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:04.995469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:05.004734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.005330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.005617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.005897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.006694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] RBAC: Adding query filter params (interface=public, service_id=7825ca080d1f45b4934b4154047cde7f, region_id=RegionOne) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:05.007008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.008406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.012073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3368ee31-1323-4606-99ba-f518c16877a9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:05.013743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 525/1031] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/endpoints?service_id=7825ca080d1f45b4934b4154047cde7f&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:05.019200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:05.046027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:05.050097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.050764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.051053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.051333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.052049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:05.052391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.053803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.066119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 507/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:05.071726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:05.143573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8621bd6-0158-4e02-b364-d74601af64a9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:05.180729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a23791-96cf-4d18-b5ae-9b35b4512f86 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:05.181709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.182646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.183823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.184140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.184845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.186229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.193746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 526/1033] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:13:05 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:13:05.218944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.219767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.220054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.220365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.229719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.231216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:13:05.247870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:05.249117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:05.255092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 508/1034] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:13:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:05.257510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:05.353131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:05.370428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d4e77-78aa-43e1-8b22-58ad00bd6a0c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:05.405626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.406527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.406965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.407413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.408446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:05.409547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.411160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.429284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 527/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:13:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:05.435204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:05.447129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:05.540840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd959f3-2e53-44b3-9733-31118eb14b0f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:05.575020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.575673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.575957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.576264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.648543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.651655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.660618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:05.757680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 509/1036] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:05.763191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:05.864180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee309b64-e611-4597-a147-acfd3cba32c7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:05.898162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:05.898957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:05.899237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:05.899509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:05.910399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:05.911858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:05.913597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-438cec69-6516-4c0a-9348-08492a115596 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:05.931502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:05.932703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:05.933572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:05.933572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:05.933572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:05.933572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:05.934231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 528/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:05.938556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:06.041845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833fb75-0226-499f-b700-18ac0b789086 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:06.051759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:06.059859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397ede5-3fe7-42ff-9c17-914010448f3e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.060164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397ede5-3fe7-42ff-9c17-914010448f3e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.060440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397ede5-3fe7-42ff-9c17-914010448f3e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.061934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 529/1038] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:06.066585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.066870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.067147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.079920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:06.081295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.081681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.081871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.082490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:06.082798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:06.083910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:06.084185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:06.085121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:06.100519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 510/1039] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:06.101528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:06.159612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 530/1040] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:13:06.183508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:06.212221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6485b-fa51-491d-a592-636bc0213afd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:06.247261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:06.248325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.248616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.248907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.256659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:06.257559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:13:06.272727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:06.274682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:06.275582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 511/1041] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:06.281558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:06.289077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030eb288-f3e8-465b-bb26-a74aeda92673 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:06.312469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:06.317994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:06.331758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:06.332612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.332891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.333170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.334420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:06.335879 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:06.337718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 531/1042] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:06.343281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:06.394897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3639324f-1bc5-4ca6-b342-c958ddb3d727 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:06.431064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:06.431648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.431932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.432243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.433042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:06.433348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:06.434775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:06.451808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 512/1043] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:06.453091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:06.453766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8431d6-ccec-4068-bc84-5031613cda12 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:06.489503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e93bbbf-655f-40c7-9416-85cdb0380199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:06.490462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93bbbf-655f-40c7-9416-85cdb0380199 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.490924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93bbbf-655f-40c7-9416-85cdb0380199 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.491203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93bbbf-655f-40c7-9416-85cdb0380199 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.491900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93bbbf-655f-40c7-9416-85cdb0380199 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:06.493417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e93bbbf-655f-40c7-9416-85cdb0380199 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:06.514863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 532/1044] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:13:06 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:13:06.566578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:06.581301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f673d91a-96ec-4858-9c4d-58224c6840e7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:06.619762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:06.620463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:06.620749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:06.621029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:06.621973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] RBAC: Adding query filter params (role.id=cf194dc6f4f04e3ba6db84118c7a069a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=37f8217df17a44a989ce1c7ab8a9ac1a) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:06.622293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:06.624064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db085d12-f9fb-4e04-8d50-2cf8e2f51a96 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:06.636131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 513/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:06 2022] GET /identity/v3/role_assignments?user.id=37f8217df17a44a989ce1c7ab8a9ac1a&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=cf194dc6f4f04e3ba6db84118c7a069a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:07.260650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa732f-499f-4472-aa38-791d3006e1a3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:07.261178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa732f-499f-4472-aa38-791d3006e1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:07.261279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fa732f-499f-4472-aa38-791d3006e1a3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:07.262618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 533/1046] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:07.267803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:07.269009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:07.269009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:07.286736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:07.287815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:07.370865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 514/1047] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:07.393381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa732f-499f-4472-aa38-791d3006e1a3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:07.444748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:07.499783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fa732f-499f-4472-aa38-791d3006e1a3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:07.532952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:07.534359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:07.534670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:07.534948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] PATH_INFO: `/v3/services/image` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:07.535604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:07.537007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:07.548570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:07.549755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 15 19:13:07.550591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:07.552403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 534/1048] 10.222.0.17 () {56 vars in 1105 bytes} [Fri Jul 15 19:13:07 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:07.555866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:07.640437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:07.660510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3756e0ac-544b-40fe-8c7c-df2680cc3915 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:07.697969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:07.698658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:07.698940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:07.699215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:07.699941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] RBAC: Adding query filter params (name=image) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:07.700281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:07.701684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:07.712273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 515/1049] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:07.713866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:07.826448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084739c-5802-4622-b91b-ffb8b0778d7a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:07.839098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:07.860470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:07.861136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:07.861422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:07.862242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:07.862946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] RBAC: Adding query filter params (type=image) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:07.863249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:07.864652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e210289-7cb9-4bc2-aca1-79275a7fd38f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:07.869652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 535/1050] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:07.875533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:07.976012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983941b3-c856-4c87-b488-856c05de4d63 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:08.009687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af124980-ce32-4672-8654-b0df8a0c34a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:08.010341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af124980-ce32-4672-8654-b0df8a0c34a2 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.010623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af124980-ce32-4672-8654-b0df8a0c34a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.010896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af124980-ce32-4672-8654-b0df8a0c34a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.011645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af124980-ce32-4672-8654-b0df8a0c34a2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:08.013030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af124980-ce32-4672-8654-b0df8a0c34a2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:08.032801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 516/1051] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:13:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:13:08.606390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0dbc-645f-402a-89da-347a14521733 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.606912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0dbc-645f-402a-89da-347a14521733 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.606912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c0dbc-645f-402a-89da-347a14521733 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.608476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 536/1052] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:08.614104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.614509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.614671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.619809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:08.633335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:08.634519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:08.653775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee1c37-e191-4bd3-a165-dca45b2677dd None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.654256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee1c37-e191-4bd3-a165-dca45b2677dd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.654601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee1c37-e191-4bd3-a165-dca45b2677dd None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.656042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 537/1053] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:08.662322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.662830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.663106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.679670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:08.680874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:08.712799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 517/1054] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:08.736796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:08.757054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 538/1055] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:08.780500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:08.853945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b46a33-aead-4b02-84f4-1afb76987345 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:08.884311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:08.889404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d51e81-accf-459b-86e9-7512d96f583b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:08.893625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:08.894415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.894693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.894963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.898365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:08.905518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:08.913687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:08.914669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:08.915536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 518/1056] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:08.920845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:08.931330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:08.932173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:08.932814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:08.933092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:08.944838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:08.952923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:08.953848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:08.954676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 539/1057] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Jul 15 19:13:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:08.960071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.038588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f00cf3-ca11-452a-be9e-35b50a40f61e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.071295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87adea-e6c3-4045-a593-ea3644415f05 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.079041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.079713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.080004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.080865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.081628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:09.081937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.083301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.093216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 519/1058] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:09.099023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.109639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.110318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.110619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.110878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.112167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:09.112474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.113922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.128119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 540/1059] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:13:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:09.130502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.152466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.166826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.209866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c904e98-f9a7-4dac-8b0b-10049fcb90c6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.244567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be254a3b-828d-4eaa-8818-726c73038e35 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.245979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.246785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.247909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.247909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] PATH_INFO: `/v3/users/baa24dde191747c4803fd1abce7d321d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.262177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] RBAC: Authorizing `identity:get_user(user_id=baa24dde191747c4803fd1abce7d321d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.263623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.270150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 520/1060] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/users/baa24dde191747c4803fd1abce7d321d => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:09.271072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.285022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.285822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.286113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.286400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.297247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.298715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:09.317056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:09.317958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:09.318844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:09.318844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:09.318844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:09.318844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:09.318844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 541/1061] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:09.323953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.375407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420db6fc-45b6-4614-83a3-5fa91ec8c175 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.385713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.406039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.415479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.416609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.416946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.417356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.428140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e442a28-2aa7-41e7-a262-88dcb46fe78f None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.464585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.465286 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.465572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.465890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.466607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:09.466911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.468286 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.487825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 542/1062] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:09.488928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.493784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.495327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.599939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400622fa-3700-4ccd-b76e-f00569a0eb4e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.605564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 521/1063] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:13:09.606470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.619881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.644240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.645034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.645317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.645591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.653004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.654463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.657583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:13:09.670831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:09.671801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:09.672860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 543/1064] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:09.678685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.729877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf929da6-981b-4d55-a3bd-3ca71bc793da None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.773740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.774637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.774883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.775160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.776838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] RBAC: Authorizing `identity:get_project(project_id=d890de0415804f05b1d0cf272f01747b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.778274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.780002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 522/1065] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/projects/d890de0415804f05b1d0cf272f01747b => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:09.786688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.789113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300221d5-9548-4f2c-bc93-599bfc049703 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.826935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.827617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.827904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.828214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.829062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:09.829374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.831153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.849304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 544/1066] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:09.854869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:09.876958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.902257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f545400-3eba-458a-98a9-8c3e651c0e49 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:09.919619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:09.938355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:09.939015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:09.939301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:09.939603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:09.940470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d890de0415804f05b1d0cf272f01747b, user.id=baa24dde191747c4803fd1abce7d321d) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:09.940791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:09.942225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:09.948029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 523/1067] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/role_assignments?user.id=baa24dde191747c4803fd1abce7d321d&scope.project.id=d890de0415804f05b1d0cf272f01747b&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:09.979117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf0b63e-4ad8-48df-82c2-05fb723373b7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.017453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.018872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.019154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.019430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.031133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.032700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-72d651bc-7fa9-45fd-9610-6d70c31c02a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:10.052287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:10.053272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:10.059285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:10.059285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:10.059285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:10.059285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:10.059285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 545/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:10.059577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.072468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf0a72-07d6-4b56-b92d-6c67e9f1e045 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.072766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf0a72-07d6-4b56-b92d-6c67e9f1e045 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.073045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bf0a72-07d6-4b56-b92d-6c67e9f1e045 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.074584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 546/1069] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:10.080844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.081783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.082108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.102143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:10.103367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:10.172326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a61e39-f873-4079-b582-7f7dc6f0f50d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.188238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 547/1070] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:10.202715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.209851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.210525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.210814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.211093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.211819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:10.212158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.213679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.225004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:10.233085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 524/1071] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:10.233942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.251320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:10.316313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43948d-2262-4fec-94de-0785613791c4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.350324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433e770-66d4-4cdc-bcd9-6e3cc4c2420b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.354771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.355761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.356266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.356717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.387566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.388397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.388683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.388965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.399842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.401754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:13:10.418063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:10.419011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:13:10.419930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:10.424325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 525/1072] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:10.425714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.433821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.435304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.479770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:10.493068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:10.538440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb984c1c-d934-4c4c-a9b8-97e121cd03e7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.546129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 548/1073] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:13:10.552580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.578510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.579173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.579459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.579736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.581110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:10.581577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.583011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.595567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 526/1074] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:10.602079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.668887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7be17-d77a-4ffc-ae06-6109b59d0c82 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.702972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.703688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.703974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.704284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.714168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:10.714999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fef86-3057-4b18-82a6-b9db589c1d2e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.716863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.718664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.736362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:10.738043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:10.739174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:10.743787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:10.743787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:10.743787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:10.743787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:10.743787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 549/1075] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:10.745876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:10.757334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00cfe6-f51f-4af9-be61-429cf0bbefb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.758234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00cfe6-f51f-4af9-be61-429cf0bbefb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.758522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00cfe6-f51f-4af9-be61-429cf0bbefb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.758803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00cfe6-f51f-4af9-be61-429cf0bbefb0 None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/37f8217df17a44a989ce1c7ab8a9ac1a/roles/cf194dc6f4f04e3ba6db84118c7a069a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.761081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00cfe6-f51f-4af9-be61-429cf0bbefb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=37f8217df17a44a989ce1c7ab8a9ac1a, role_id=cf194dc6f4f04e3ba6db84118c7a069a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.762526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00cfe6-f51f-4af9-be61-429cf0bbefb0 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.779859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 527/1076] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:13:10 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/37f8217df17a44a989ce1c7ab8a9ac1a/roles/cf194dc6f4f04e3ba6db84118c7a069a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:13:10.857273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5224464f-22dc-4127-b06a-b2abf39a2e4b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:10.894822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:10.895507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:10.895815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:10.896114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:10.896860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:10.897168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:10.898695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c14c8ea-2f70-479e-a1c6-d667a08a1a2a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:10.920127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 550/1077] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:11.937378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ac9f9-7d8d-44b1-b78a-d9d85fbb90eb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:11.937939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ac9f9-7d8d-44b1-b78a-d9d85fbb90eb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:11.937939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ac9f9-7d8d-44b1-b78a-d9d85fbb90eb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:11.939299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 528/1078] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:11.945406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:11.945815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:11.946090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:11.963799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:11.965008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:12.009666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:12.029693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 551/1079] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:12.051886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ac9f9-7d8d-44b1-b78a-d9d85fbb90eb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:12.152304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ac9f9-7d8d-44b1-b78a-d9d85fbb90eb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:12.186537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:12.187322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.187607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.187879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.198705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:12.206989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:12.208072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:12.214079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 529/1080] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:12.214773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:12.311251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ec448-efc5-4d53-ac23-7395d310aa79 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:12.343700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:12.344404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.344687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.344959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.345717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:12.346602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:12.347975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:12.355959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 552/1081] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:12.361956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:12.403505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:12.463147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442c3799-a6b3-45a9-95ea-e08b9e391e9b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:12.496984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:12.498115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.498397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.498669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] PATH_INFO: `/v3/users/baa24dde191747c4803fd1abce7d321d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.499860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] RBAC: Authorizing `identity:get_user(user_id=baa24dde191747c4803fd1abce7d321d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:12.501346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:12.508640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 530/1082] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:12 2022] GET /identity/v3/users/baa24dde191747c4803fd1abce7d321d => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:12.509798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:12.554644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:12.608225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f335fb6-4e74-4a7c-a5f7-a254f2e20a18 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:12.641343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d07518c-25a9-4860-a247-5c27e68fb118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:12.642199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07518c-25a9-4860-a247-5c27e68fb118 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.642482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07518c-25a9-4860-a247-5c27e68fb118 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.643358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d07518c-25a9-4860-a247-5c27e68fb118 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.644643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07518c-25a9-4860-a247-5c27e68fb118 None admin] RBAC: Authorizing `identity:get_project(project_id=d890de0415804f05b1d0cf272f01747b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:12.646077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d07518c-25a9-4860-a247-5c27e68fb118 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:12.647762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 553/1083] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:12 2022] GET /identity/v3/projects/d890de0415804f05b1d0cf272f01747b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:12.653956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:12.760578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c297195c-f2ee-4f7e-acd0-3f81ab6ab1ef None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:12.770743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:12.795410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942980d7-2332-4f84-9d6f-3e8973b7fb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:12.796322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942980d7-2332-4f84-9d6f-3e8973b7fb42 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.796605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942980d7-2332-4f84-9d6f-3e8973b7fb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.796877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942980d7-2332-4f84-9d6f-3e8973b7fb42 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b/users/baa24dde191747c4803fd1abce7d321d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.798836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942980d7-2332-4f84-9d6f-3e8973b7fb42 None admin] RBAC: Authorizing `identity:create_grant(project_id=d890de0415804f05b1d0cf272f01747b, user_id=baa24dde191747c4803fd1abce7d321d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:12.800345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942980d7-2332-4f84-9d6f-3e8973b7fb42 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:12.813041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 531/1084] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:13:12 2022] PUT /identity/v3/projects/d890de0415804f05b1d0cf272f01747b/users/baa24dde191747c4803fd1abce7d321d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:13:12.854207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda711d-3f4b-4812-b7d2-b5f2236ef075 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.854502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda711d-3f4b-4812-b7d2-b5f2236ef075 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.854788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda711d-3f4b-4812-b7d2-b5f2236ef075 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.856350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 554/1085] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:12.862468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:12.862678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:12.862959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:12.881415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:12.882650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:12.954329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 532/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:12.977059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda711d-3f4b-4812-b7d2-b5f2236ef075 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:13.078964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda711d-3f4b-4812-b7d2-b5f2236ef075 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:13.091659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:13.114319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:13.115052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:13.115334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:13.115610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:13.126370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:13.134763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:13.135811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:13.136917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 555/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Fri Jul 15 19:13:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:13.141921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:13.240553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:13.244181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb6585-1518-4859-b698-b7e062cd5568 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:13.280530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:13.281199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:13.281481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:13.281756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:13.282505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:13.282813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:13.284274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:13.292245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 533/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:13:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:13.297783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:13.394811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64e0b8-067c-4fa3-8e42-9a66421f47f9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:13.427726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:13.428544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:13.428827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:13.429103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:13.439119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:13.441134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:13.458664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:13.459525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:13.460436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:13.460436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:13.460436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:13.460436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:13.460436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 556/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:13.465550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:13.563396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d8c01-ae59-428e-9f59-3a406391112b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:13.596584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:13.597235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:13.597517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:13.597795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:13.598509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:13.598816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:13.600198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:13.611762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 534/1090] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:13.617238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:13.643889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:13.716972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690e532f-7b2a-453f-bd4d-ed8cc974e3d1 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:13.751852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:13.753308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:13.753591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:13.753864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:13.761627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:13.763056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:13:13.776600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:13.777715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:13.778689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 557/1091] 10.222.0.17 () {56 vars in 1093 bytes} [Fri Jul 15 19:13:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:13.783545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:13.805127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:13.883504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d2553-8546-4c8b-9c2a-10eb4bffaeef None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:13.918131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:13.918831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:13.919114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:13.919410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:13.920251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:13.920588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:13.922012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:13.940211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 535/1092] 10.222.0.17 () {56 vars in 1133 bytes} [Fri Jul 15 19:13:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:13.945587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:14.028156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:14.045226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6c73-df33-4af8-8104-d83cdbb4ec16 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:14.078607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:14.079251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.079535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.079814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.145250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:14.146698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:14.226429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:14.243054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 558/1093] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:13:14.248923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:14.326430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:14.348902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705f413-77d4-4db8-b5e5-6bc1c04c7407 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:14.383355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:14.384168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.384454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.384732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.395194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:14.396683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:14.416510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:14.417697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:14.418610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:13:14.418610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:14.418610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:13:14.418610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:14.418610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 536/1094] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:14.423505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:14.521047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac266b8-18ad-4657-beeb-95b9b942eeb6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:14.554340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:14.554997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.555279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.555555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.556303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:14.556610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:14.557989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:14.569232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 559/1095] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:14.574803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:14.676027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ea5e2-3151-4538-b709-55242a035777 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:14.706693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:14.712636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:14.713424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.713705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.713976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.721201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:14.723032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-616fe3d2-73bf-4197-819a-d7ac32e460cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:13:14.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:14.738262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:14.739196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 537/1096] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:13:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:14.744057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:14.791034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a95bd7-38d5-4f78-b262-055317265cea None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.791207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:14.791300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a95bd7-38d5-4f78-b262-055317265cea None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.791589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a95bd7-38d5-4f78-b262-055317265cea None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.793192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 538/1097] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:13:14.798241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.798524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.798802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.816012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:14.817223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:14.846355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339ba24-ec25-479e-8225-a1bebdcbab0a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:14.878899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:14.879556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:14.879839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:14.880132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:14.880913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:14.881218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:14.882646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:14.883421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 539/1098] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:13:14.895288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 560/1099] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:13:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:14.901207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:14.905871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:15.001166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3c70b-dd90-49e0-921c-be7cc893efd9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:15.004324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ee480-7c75-41cc-86de-b6fc9ba5cf23 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:15.006567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:15.021879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:15.036405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:15.037061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:15.037346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:15.037621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:15.038447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] RBAC: Adding query filter params (role.id=cf194dc6f4f04e3ba6db84118c7a069a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=37f8217df17a44a989ce1c7ab8a9ac1a) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:15.038758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:15.039308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:15.040113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:15.040211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:15.040405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:15.040681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:15.046263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 540/1100] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:14 2022] GET /identity/v3/role_assignments?user.id=37f8217df17a44a989ce1c7ab8a9ac1a&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=cf194dc6f4f04e3ba6db84118c7a069a => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:15.051635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:15.059530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:15.060756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:15.062008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 561/1101] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:13:15.067206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:15.178195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c81ea-063f-4769-95b8-05d164c44634 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:15.220233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:15.221939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:15.222469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:15.222943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:15.223888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:15.224424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:15.226007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:15.236963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 541/1102] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:15.243219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:15.366251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb792859-6abb-4275-819a-0aabc6c163eb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:15.410099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:15.411010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:15.411292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:15.411565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] PATH_INFO: `/v3/users/baa24dde191747c4803fd1abce7d321d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:15.412923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] RBAC: Authorizing `identity:get_user(user_id=baa24dde191747c4803fd1abce7d321d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:15.414514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:15.416698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 562/1103] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:15 2022] GET /identity/v3/users/baa24dde191747c4803fd1abce7d321d => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:15.423289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:15.483339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:15.538644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fa73f1-3dad-4b4e-ace9-b4db1081dfa5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:15.572930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:15.573585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:15.573870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:15.574148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:15.642236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:15.645079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:15.698699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:15.758897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 542/1104] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:15.765407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:15.797762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:15.869917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c3a08-92bb-47d0-b529-30dd9d430394 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:15.903740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadab4c-f397-479b-9821-61bfd298aa18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:15.904554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadab4c-f397-479b-9821-61bfd298aa18 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:15.904832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadab4c-f397-479b-9821-61bfd298aa18 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:15.905106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadab4c-f397-479b-9821-61bfd298aa18 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:15.906276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadab4c-f397-479b-9821-61bfd298aa18 None admin] RBAC: Authorizing `identity:get_project(project_id=d890de0415804f05b1d0cf272f01747b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:15.907713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadab4c-f397-479b-9821-61bfd298aa18 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:15.910145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 563/1105] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:15 2022] GET /identity/v3/projects/d890de0415804f05b1d0cf272f01747b => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:15.916276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:16.014439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07645ac4-e880-4f6d-a860-cc6eba1042e3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:16.048089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:16.048776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:16.049110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:16.049334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:16.050158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d890de0415804f05b1d0cf272f01747b, user.id=baa24dde191747c4803fd1abce7d321d) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:16.050468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:16.051904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a63916-5721-4f6e-8e12-1b34dd17617f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:16.057680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 543/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:15 2022] GET /identity/v3/role_assignments?user.id=baa24dde191747c4803fd1abce7d321d&scope.project.id=d890de0415804f05b1d0cf272f01747b&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:18.066136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984b7a9-d4b2-4c44-8b1e-2829e12720d3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:18.066647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984b7a9-d4b2-4c44-8b1e-2829e12720d3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:18.066647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2984b7a9-d4b2-4c44-8b1e-2829e12720d3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:18.067964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 564/1107] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:18.073221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:18.073622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:18.073860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:18.091017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:18.092236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:18.094918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:18.157609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 544/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:18.181921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984b7a9-d4b2-4c44-8b1e-2829e12720d3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:18.279416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:18.284641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2984b7a9-d4b2-4c44-8b1e-2829e12720d3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:18.318408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadc140-fc37-4978-97af-44a023b9fc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:18.319138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadc140-fc37-4978-97af-44a023b9fc0a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:18.319425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadc140-fc37-4978-97af-44a023b9fc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:18.319704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadc140-fc37-4978-97af-44a023b9fc0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:18.320906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadc140-fc37-4978-97af-44a023b9fc0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:18.322342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eadc140-fc37-4978-97af-44a023b9fc0a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:18.323943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 565/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:18.330473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:18.430142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be24205-659c-41f2-b7a0-9d18148a0338 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:18.462822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160d44c6-3c29-4195-b859-407f115bd1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:18.463481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d44c6-3c29-4195-b859-407f115bd1e6 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:18.463760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d44c6-3c29-4195-b859-407f115bd1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:18.464030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160d44c6-3c29-4195-b859-407f115bd1e6 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:18.464752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d44c6-3c29-4195-b859-407f115bd1e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:18.466269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160d44c6-3c29-4195-b859-407f115bd1e6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:18.476264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:18.487198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 545/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:13:18 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:13:20.583889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbe9e3-7853-4bbc-88c2-3c21330d8be0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:20.584564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbe9e3-7853-4bbc-88c2-3c21330d8be0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:20.584564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bbe9e3-7853-4bbc-88c2-3c21330d8be0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:20.585842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 566/1111] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:20.590965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:20.591389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:20.591528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:20.608673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:20.609861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:20.675441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 546/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:20.698564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbe9e3-7853-4bbc-88c2-3c21330d8be0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:20.798371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bbe9e3-7853-4bbc-88c2-3c21330d8be0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:20.831036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:20.831882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:20.832185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:20.832461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] PATH_INFO: `/v3/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:20.833963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] RBAC: Authorizing `identity:get_role(role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:20.835275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:20.836653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 567/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:20 2022] GET /identity/v3/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:20.842579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:20.938016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88794d5-f075-48fe-a00d-86a4da785cd8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:20.963899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:20.972900 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:20.973764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:20.974045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:20.974321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] PATH_INFO: `/v3/users/5b18028ef59b4204b93efb0c2d914822` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:20.986933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] RBAC: Authorizing `identity:get_user(user_id=5b18028ef59b4204b93efb0c2d914822)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:20.988396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:20.990034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 547/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:20 2022] GET /identity/v3/users/5b18028ef59b4204b93efb0c2d914822 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:20.995706 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:21.030723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:21.092810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438bb48-c176-42f0-96bb-bc6f94367cc2 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:21.124627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:21.125255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:21.125534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:21.125811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:21.190022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:21.191451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:21.222629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:21.284883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 568/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:21.290728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:21.388275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dba05-b0f1-4cb0-aaf2-ee6ea8871366 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:21.421243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34f9808-beb1-47d0-af60-17dada632a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:21.422183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f9808-beb1-47d0-af60-17dada632a09 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:21.422476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f9808-beb1-47d0-af60-17dada632a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:21.422899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34f9808-beb1-47d0-af60-17dada632a09 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:21.424956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f9808-beb1-47d0-af60-17dada632a09 None admin] RBAC: Authorizing `identity:get_project(project_id=d890de0415804f05b1d0cf272f01747b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:21.426552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34f9808-beb1-47d0-af60-17dada632a09 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:21.429033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 548/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:21 2022] GET /identity/v3/projects/d890de0415804f05b1d0cf272f01747b => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:21.435142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:21.530740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a6a6df-0b30-4d01-82cb-63f41a3c8747 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:21.563709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:21.564551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:21.564832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:21.565101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:21.565914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] RBAC: Adding query filter params (role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=d890de0415804f05b1d0cf272f01747b, user.id=5b18028ef59b4204b93efb0c2d914822) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:21.566220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:21.567641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9801cf-9599-430a-af85-f0d02df4f817 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:21.571160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:21.576802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 569/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:21 2022] GET /identity/v3/role_assignments?user.id=5b18028ef59b4204b93efb0c2d914822&scope.project.id=d890de0415804f05b1d0cf272f01747b&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:23.539606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4fa3c-48ab-4b16-9b42-266de3435f57 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:23.540247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4fa3c-48ab-4b16-9b42-266de3435f57 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:23.540247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4fa3c-48ab-4b16-9b42-266de3435f57 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:23.541564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 549/1118] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:23.546908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:23.547341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:23.547460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:23.565185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:23.566357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:23.629980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 570/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:23.651146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4fa3c-48ab-4b16-9b42-266de3435f57 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:23.703675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:23.751515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4fa3c-48ab-4b16-9b42-266de3435f57 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:23.785415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:23.786271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:23.786554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:23.786829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] PATH_INFO: `/v3/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:23.788331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] RBAC: Authorizing `identity:get_role(role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:23.789747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:23.791081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 550/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:23 2022] GET /identity/v3/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:23.796812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:23.891496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d83a1e3-eb5f-4755-a981-54dde52e7879 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:23.918673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:23.924770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:23.925631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:23.925911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:23.926182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] PATH_INFO: `/v3/users/5b18028ef59b4204b93efb0c2d914822` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:23.927347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] RBAC: Authorizing `identity:get_user(user_id=5b18028ef59b4204b93efb0c2d914822)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:23.928811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:23.930402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 571/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:23 2022] GET /identity/v3/users/5b18028ef59b4204b93efb0c2d914822 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:23.936195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:24.034144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa5cf8c-d17a-4f8b-9acd-54702977ce3a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:24.054063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:24.068340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c3e30-8429-467e-ba77-47dc86ba4022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:24.069188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3e30-8429-467e-ba77-47dc86ba4022 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:24.069468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3e30-8429-467e-ba77-47dc86ba4022 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:24.069739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c3e30-8429-467e-ba77-47dc86ba4022 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:24.070890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3e30-8429-467e-ba77-47dc86ba4022 None admin] RBAC: Authorizing `identity:get_project(project_id=d890de0415804f05b1d0cf272f01747b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:24.072328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c3e30-8429-467e-ba77-47dc86ba4022 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:24.074008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 551/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:23 2022] GET /identity/v3/projects/d890de0415804f05b1d0cf272f01747b => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:24.080576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:24.181535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943fa85b-43b1-41b3-b513-34e4b96d91e8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:24.214516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2203a-e4ed-45b7-9aaf-b880cbbc8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:24.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2203a-e4ed-45b7-9aaf-b880cbbc8a25 None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:24.215693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2203a-e4ed-45b7-9aaf-b880cbbc8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:24.215970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2203a-e4ed-45b7-9aaf-b880cbbc8a25 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b/users/5b18028ef59b4204b93efb0c2d914822/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:24.218247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2203a-e4ed-45b7-9aaf-b880cbbc8a25 None admin] RBAC: Authorizing `identity:create_grant(project_id=d890de0415804f05b1d0cf272f01747b, user_id=5b18028ef59b4204b93efb0c2d914822, role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:24.219693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2203a-e4ed-45b7-9aaf-b880cbbc8a25 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:24.232481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 572/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:13:24 2022] PUT /identity/v3/projects/d890de0415804f05b1d0cf272f01747b/users/5b18028ef59b4204b93efb0c2d914822/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:13:26.293231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb05b2d-20dd-463a-a860-be6f304c01cb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:26.293782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb05b2d-20dd-463a-a860-be6f304c01cb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:26.293782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb05b2d-20dd-463a-a860-be6f304c01cb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:26.295150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 552/1124] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:26.301278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:26.301687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:26.301838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:26.318866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:26.320042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:26.371669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:26.386145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 573/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:26.410419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb05b2d-20dd-463a-a860-be6f304c01cb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:26.511358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb05b2d-20dd-463a-a860-be6f304c01cb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:26.545873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:26.546647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:26.546934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:26.547214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] PATH_INFO: `/v3/roles/1b6a42dfe81b458fae2015e8425508a8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:26.549176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] RBAC: Authorizing `identity:get_role(role_id=1b6a42dfe81b458fae2015e8425508a8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:26.550537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:26.551898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 553/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:26 2022] GET /identity/v3/roles/1b6a42dfe81b458fae2015e8425508a8 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:26.557672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:26.652743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b36f5-5723-4f88-bfd8-8774742226fd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:26.685000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:26.685854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:26.686136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:26.686410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] PATH_INFO: `/v3/users/5b18028ef59b4204b93efb0c2d914822` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:26.687582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] RBAC: Authorizing `identity:get_user(user_id=5b18028ef59b4204b93efb0c2d914822)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:26.690091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:26.698984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 574/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:26 2022] GET /identity/v3/users/5b18028ef59b4204b93efb0c2d914822 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:26.704826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:26.761747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:26.805512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c7d7b-92d5-498a-964f-c48124ee7062 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:26.839050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:26.839696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:26.839982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:26.840287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:26.905398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:26.906843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:26.959371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:27.003473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 554/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:27.009362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:27.064742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:27.107015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b0aaae-482f-4873-9b5a-0a5c90bf0ea7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:27.139266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ca5352-a4b7-435c-8711-3531213ff901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:27.140125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5352-a4b7-435c-8711-3531213ff901 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:27.140414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5352-a4b7-435c-8711-3531213ff901 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:27.140687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ca5352-a4b7-435c-8711-3531213ff901 None admin] PATH_INFO: `/v3/projects/d890de0415804f05b1d0cf272f01747b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:27.142512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca5352-a4b7-435c-8711-3531213ff901 None admin] RBAC: Authorizing `identity:get_project(project_id=d890de0415804f05b1d0cf272f01747b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:27.143953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ca5352-a4b7-435c-8711-3531213ff901 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:27.145670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 575/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:27 2022] GET /identity/v3/projects/d890de0415804f05b1d0cf272f01747b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:27.151737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:27.247540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265caa9-e903-4a0d-b868-d83b5befb945 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:27.280161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:27.280799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:27.281082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:27.281357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:27.282178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] RBAC: Adding query filter params (role.id=1b6a42dfe81b458fae2015e8425508a8, scope.project.id=d890de0415804f05b1d0cf272f01747b, user.id=5b18028ef59b4204b93efb0c2d914822) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:27.282589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:27.284017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda981e-7a21-4c9c-a6dc-84b9810410fa None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:27.289751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 555/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:27 2022] GET /identity/v3/role_assignments?user.id=5b18028ef59b4204b93efb0c2d914822&scope.project.id=d890de0415804f05b1d0cf272f01747b&role.id=1b6a42dfe81b458fae2015e8425508a8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:29.296350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c98f5-a5b2-430c-833f-98d98226dbf4 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:29.296897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c98f5-a5b2-430c-833f-98d98226dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:29.296897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c98f5-a5b2-430c-833f-98d98226dbf4 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:29.299010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 576/1131] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:29.304319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:29.304735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:29.304882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:29.321355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:29.322543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:29.336731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:29.388509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 556/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:29.412716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c98f5-a5b2-430c-833f-98d98226dbf4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:29.513192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c98f5-a5b2-430c-833f-98d98226dbf4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:29.536549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:29.547289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef790d6-ebf5-4d03-bdc8-188da4106279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:29.548044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef790d6-ebf5-4d03-bdc8-188da4106279 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:29.548354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef790d6-ebf5-4d03-bdc8-188da4106279 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:29.548632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef790d6-ebf5-4d03-bdc8-188da4106279 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:29.549789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef790d6-ebf5-4d03-bdc8-188da4106279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:29.551214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef790d6-ebf5-4d03-bdc8-188da4106279 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:29.552847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 577/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:29.558953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:29.654521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c432aa-c80a-4464-b5de-ed95ae4a3b53 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:29.687169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dab9fd-7623-4448-adcd-b2705b12807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:29.687811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dab9fd-7623-4448-adcd-b2705b12807a None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:29.688115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dab9fd-7623-4448-adcd-b2705b12807a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:29.688402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dab9fd-7623-4448-adcd-b2705b12807a None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:29.689515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dab9fd-7623-4448-adcd-b2705b12807a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:29.690941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dab9fd-7623-4448-adcd-b2705b12807a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:29.715885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 557/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:13:29 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:13:31.724858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf6730-8c77-49ab-b108-eebc4ef791b8 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:31.725388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf6730-8c77-49ab-b108-eebc4ef791b8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:31.725388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf6730-8c77-49ab-b108-eebc4ef791b8 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:31.726800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 578/1135] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:31.732134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:31.732514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:31.732691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:31.737676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:31.751893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:31.753102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:31.818427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 558/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:31.842591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf6730-8c77-49ab-b108-eebc4ef791b8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:31.944006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf6730-8c77-49ab-b108-eebc4ef791b8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:31.976801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b78eb-540c-495a-8ead-823e2fa50dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:31.977508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b78eb-540c-495a-8ead-823e2fa50dcb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:31.977788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b78eb-540c-495a-8ead-823e2fa50dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:31.978061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b78eb-540c-495a-8ead-823e2fa50dcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:31.979206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b78eb-540c-495a-8ead-823e2fa50dcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:31.980671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b78eb-540c-495a-8ead-823e2fa50dcb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:31.982255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 579/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:13:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:31.988376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:32.083854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f752b7-4fb8-4225-b896-63edc8ecfea6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:32.112249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:32.117468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183daf0-4937-476c-b921-f393e12ccf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:32.118127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183daf0-4937-476c-b921-f393e12ccf89 None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:32.118407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183daf0-4937-476c-b921-f393e12ccf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:32.118682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183daf0-4937-476c-b921-f393e12ccf89 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:32.119392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183daf0-4937-476c-b921-f393e12ccf89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:32.120839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183daf0-4937-476c-b921-f393e12ccf89 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:32.139599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 559/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:13:31 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:13:34.268429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ba21-eda2-4c74-970f-a60cda786604 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:34.268989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ba21-eda2-4c74-970f-a60cda786604 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:34.268989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564ba21-eda2-4c74-970f-a60cda786604 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:34.270321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 580/1139] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:34.275405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:34.275822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:34.275970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:34.292838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:34.294014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:34.357780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 560/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:34.382434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ba21-eda2-4c74-970f-a60cda786604 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:34.440911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:34.481131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564ba21-eda2-4c74-970f-a60cda786604 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:34.513864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:34.514646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:34.514929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:34.515206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:34.525291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:34.533385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:34.534554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:34.535499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 581/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:34.540477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:34.635242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81611b-e1f9-4d0f-8bdc-e023120c0ea2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:34.652176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:34.669550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:34.670211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:34.670493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:34.670769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:34.671516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:34.671823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:34.673262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:34.681112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 561/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:34.686889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:34.786564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a48fe22-a083-47c2-b8de-0536d54c63c5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:34.805518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:34.819309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:34.820201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:34.820482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:34.820756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] PATH_INFO: `/v3/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:34.834038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=a9a4bf4ef67d4c389dcdf31ca72b9cb7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:34.835472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:34.837145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 582/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:34 2022] GET /identity/v3/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:34.843012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:34.943421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89473f67-6b88-4a73-8b53-dd140e24e837 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:34.976924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:34.977565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:34.977851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:34.978127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:35.025443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:35.044157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:35.045550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:35.145351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 562/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:35.151243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:35.245488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4d8b8-df22-4c82-892f-d20b1f26d2a5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:35.278633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d37153-7e4e-4946-826a-ae0a03684114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:35.279482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37153-7e4e-4946-826a-ae0a03684114 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:35.279764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37153-7e4e-4946-826a-ae0a03684114 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:35.280040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37153-7e4e-4946-826a-ae0a03684114 None admin] PATH_INFO: `/v3/projects/a91c4b2acd8447efaab8d139511c0840` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:35.281234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d37153-7e4e-4946-826a-ae0a03684114 None admin] RBAC: Authorizing `identity:get_project(project_id=a91c4b2acd8447efaab8d139511c0840)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:35.282735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d37153-7e4e-4946-826a-ae0a03684114 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:35.284447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 583/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:35 2022] GET /identity/v3/projects/a91c4b2acd8447efaab8d139511c0840 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:35.290364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:35.367950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:35.387147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfca7de-5aa8-4770-8321-ac94f85914e7 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:35.419166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:35.419809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:35.420114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:35.420394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:35.421214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=a91c4b2acd8447efaab8d139511c0840, user.id=a9a4bf4ef67d4c389dcdf31ca72b9cb7) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:35.421520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:35.423458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98098a98-1d7a-4d6b-85ce-2d38ce206b70 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:35.428771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 563/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:35 2022] GET /identity/v3/role_assignments?user.id=a9a4bf4ef67d4c389dcdf31ca72b9cb7&scope.project.id=a91c4b2acd8447efaab8d139511c0840&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:37.385683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f4212-280f-4124-ab96-2d27ff438028 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:37.386210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f4212-280f-4124-ab96-2d27ff438028 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:37.386210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f4212-280f-4124-ab96-2d27ff438028 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:37.387582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 584/1147] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:37.392886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:37.393302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:37.393439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:37.410121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:37.411281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:37.477964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 564/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:37.500978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f4212-280f-4124-ab96-2d27ff438028 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:37.547139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:37.598956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f4212-280f-4124-ab96-2d27ff438028 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:37.631222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:37.631991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:37.632293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:37.632574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:37.643006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:37.650622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:37.651731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:37.652801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 585/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:37.658489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:37.729729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:37.754338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76a178e-2f9f-4745-aa10-1f7ea6d2b8bf None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:37.788411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:37.789085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:37.789369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:37.789644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:37.790395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:37.790705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:37.792107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:37.801439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 565/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:37.807248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:37.904323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5f236-4934-47f6-8d34-76f6e812ac67 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:37.911331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:37.937666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:37.938552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:37.938833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:37.939107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] PATH_INFO: `/v3/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:37.940297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] RBAC: Authorizing `identity:get_user(user_id=a9a4bf4ef67d4c389dcdf31ca72b9cb7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:37.941720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:37.943309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 586/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:37 2022] GET /identity/v3/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:37.949022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:38.048929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afea642d-9773-4296-aa09-d4c037f0d892 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:38.081375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e20b9-5390-412c-9ace-302118fd4253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:38.082230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e20b9-5390-412c-9ace-302118fd4253 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:38.082513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e20b9-5390-412c-9ace-302118fd4253 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:38.082790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e20b9-5390-412c-9ace-302118fd4253 None admin] PATH_INFO: `/v3/projects/a91c4b2acd8447efaab8d139511c0840` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:38.083953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e20b9-5390-412c-9ace-302118fd4253 None admin] RBAC: Authorizing `identity:get_project(project_id=a91c4b2acd8447efaab8d139511c0840)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:38.085445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760e20b9-5390-412c-9ace-302118fd4253 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:38.087144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 566/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:37 2022] GET /identity/v3/projects/a91c4b2acd8447efaab8d139511c0840 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:38.093409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:38.197864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77046f65-a845-4064-a16c-079fbf81e0e8 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:38.231441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65803fb-0bf9-487f-9e00-9af35768ed4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:38.232347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65803fb-0bf9-487f-9e00-9af35768ed4b None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:38.232627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65803fb-0bf9-487f-9e00-9af35768ed4b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:38.232898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65803fb-0bf9-487f-9e00-9af35768ed4b None admin] PATH_INFO: `/v3/projects/a91c4b2acd8447efaab8d139511c0840/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:38.234822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65803fb-0bf9-487f-9e00-9af35768ed4b None admin] RBAC: Authorizing `identity:create_grant(project_id=a91c4b2acd8447efaab8d139511c0840, user_id=a9a4bf4ef67d4c389dcdf31ca72b9cb7, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:38.237148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65803fb-0bf9-487f-9e00-9af35768ed4b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:38.251234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 587/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:13:38 2022] PUT /identity/v3/projects/a91c4b2acd8447efaab8d139511c0840/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:13:40.212142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2bb5d-1274-43b2-83c0-131536c9f2a0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:40.212668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2bb5d-1274-43b2-83c0-131536c9f2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:40.212668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d2bb5d-1274-43b2-83c0-131536c9f2a0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:40.214053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 567/1154] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:40.219719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:40.220189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:40.220298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:40.237050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:40.238215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:40.263699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:40.304305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 588/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:40.329540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2bb5d-1274-43b2-83c0-131536c9f2a0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:40.333268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:40.430601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d2bb5d-1274-43b2-83c0-131536c9f2a0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:40.462623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:40.463346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:40.463629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:40.463907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:40.474842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:40.482349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:40.483379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:40.484267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 568/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:40.490227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:40.584564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da0fe0-66a0-40c3-9a47-a9a51d7702aa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:40.617009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:40.617659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:40.617937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:40.618209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:40.618942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:40.619242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:40.620633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:40.628656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 589/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:40.634274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:40.695802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:40.733229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d97e6-0208-4962-aa92-b2ca9471f7e6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:40.766939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:40.767794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:40.768098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:40.768387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] PATH_INFO: `/v3/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:40.769592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] RBAC: Authorizing `identity:get_user(user_id=a9a4bf4ef67d4c389dcdf31ca72b9cb7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:40.771032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:40.772705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 569/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:40 2022] GET /identity/v3/users/a9a4bf4ef67d4c389dcdf31ca72b9cb7 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:40.778502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:40.805712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:40.875231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798248d6-52c3-419c-96ea-e6e168b7c3a6 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:40.907576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:40.908246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:40.908529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:40.908805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:40.973139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:40.974572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:40.999109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:41.069768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 590/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:41.075757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:41.171567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064d89f-3ebb-4bbc-9175-0ed67f18f7af None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:41.199715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:41.205009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe8667-a1a2-41c5-bf34-722a0a189d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:41.205851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe8667-a1a2-41c5-bf34-722a0a189d31 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:41.206134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe8667-a1a2-41c5-bf34-722a0a189d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:41.206409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe8667-a1a2-41c5-bf34-722a0a189d31 None admin] PATH_INFO: `/v3/projects/a91c4b2acd8447efaab8d139511c0840` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:41.207571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe8667-a1a2-41c5-bf34-722a0a189d31 None admin] RBAC: Authorizing `identity:get_project(project_id=a91c4b2acd8447efaab8d139511c0840)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:41.209041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe8667-a1a2-41c5-bf34-722a0a189d31 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:41.210712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 570/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:41 2022] GET /identity/v3/projects/a91c4b2acd8447efaab8d139511c0840 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:41.216722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:41.313187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f328b666-3a3e-4bae-b78c-abd0913e46ee None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:41.337779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:41.346297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:41.346934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:41.347219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:41.347495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:41.348343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=a91c4b2acd8447efaab8d139511c0840, user.id=a9a4bf4ef67d4c389dcdf31ca72b9cb7) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:41.348656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:41.350080 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ccbc0-87ef-4d60-8f40-486e29306d5d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:41.355710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 591/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:41 2022] GET /identity/v3/role_assignments?user.id=a9a4bf4ef67d4c389dcdf31ca72b9cb7&scope.project.id=a91c4b2acd8447efaab8d139511c0840&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:43.348740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec13897-7169-475f-a0c2-7eb6f66547ab None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:43.349291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec13897-7169-475f-a0c2-7eb6f66547ab None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:43.349291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec13897-7169-475f-a0c2-7eb6f66547ab None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:43.350677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 571/1162] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:43.355689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:43.356101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:43.356328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:43.373438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:43.374595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:43.438965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 592/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:43.460511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec13897-7169-475f-a0c2-7eb6f66547ab None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:43.559691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec13897-7169-475f-a0c2-7eb6f66547ab None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:43.592630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:43.593258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:43.593538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:43.593809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:43.655803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:43.658180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:43.659923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:43.755127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 572/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:43.760669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:43.855762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71793d79-35bd-4c25-8880-957db9540c04 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:43.875423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:43.891464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:43.892197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:43.892480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:43.892751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:43.899210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:43.900697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b90876dc-e8d9-4910-a6f4-f4ebb4b8d220 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:43.914721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:43.915889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:43.917019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 593/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:13:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:43.922019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:44.014636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:44.018093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bd943-612d-4c67-ac1c-2e5728d9e488 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:44.053476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:44.054189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:44.054471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:44.054745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:44.055525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:44.055827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:44.057204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0137a68e-bd88-47a6-b3fd-5235acf9e8ed None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:44.064193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 573/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Fri Jul 15 19:13:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:46.053935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc3d30-685d-4a00-8593-a1cd230bfad2 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:46.054454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc3d30-685d-4a00-8593-a1cd230bfad2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:46.054454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dc3d30-685d-4a00-8593-a1cd230bfad2 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:46.055760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 594/1167] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:46.060907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b670e-7d49-456c-a857-e05dce0b4b54 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:46.061287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b670e-7d49-456c-a857-e05dce0b4b54 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:46.061468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b670e-7d49-456c-a857-e05dce0b4b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:46.077886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-933b670e-7d49-456c-a857-e05dce0b4b54 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:46.079581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-933b670e-7d49-456c-a857-e05dce0b4b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:46.144845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 574/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 15 19:13:46.167252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc3d30-685d-4a00-8593-a1cd230bfad2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:46.266223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dc3d30-685d-4a00-8593-a1cd230bfad2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:46.299144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ee5ca-5911-4361-8713-c0f0d251a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:46.299793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee5ca-5911-4361-8713-c0f0d251a0ad None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:46.300103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee5ca-5911-4361-8713-c0f0d251a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:46.300389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ee5ca-5911-4361-8713-c0f0d251a0ad None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:46.301128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ee5ca-5911-4361-8713-c0f0d251a0ad None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:46.302500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4ee5ca-5911-4361-8713-c0f0d251a0ad None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:46.317089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 595/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Fri Jul 15 19:13:46 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:13:48.342242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2410-c6ba-4e23-bb06-69a0920d75fb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:48.342789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2410-c6ba-4e23-bb06-69a0920d75fb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:48.342789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe2410-c6ba-4e23-bb06-69a0920d75fb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:48.344365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 575/1170] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:48.349496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:48.349920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:48.350058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:48.366909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:48.368107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:48.401301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:48.434152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 596/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:48.457611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe2410-c6ba-4e23-bb06-69a0920d75fb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:48.549492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:48.557389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbe2410-c6ba-4e23-bb06-69a0920d75fb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:48.589184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2ca60-6875-45ec-ae6d-031531333d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:48.590026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2ca60-6875-45ec-ae6d-031531333d3a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:48.590315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2ca60-6875-45ec-ae6d-031531333d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:48.590588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2ca60-6875-45ec-ae6d-031531333d3a None admin] PATH_INFO: `/v3/domains/5201235c0684489ea6d2258db1c1c15f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:48.601258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2ca60-6875-45ec-ae6d-031531333d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=5201235c0684489ea6d2258db1c1c15f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:48.602647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2ca60-6875-45ec-ae6d-031531333d3a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:48.604296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 576/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Jul 15 19:13:48 2022] GET /identity/v3/domains/5201235c0684489ea6d2258db1c1c15f => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:48.610799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:48.709859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253abee-5bd4-4387-a48b-c4bf4dea159f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:48.742988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8aa9bd-dd48-41aa-b915-2f390ab508e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:48.743651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8aa9bd-dd48-41aa-b915-2f390ab508e3 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:48.743932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8aa9bd-dd48-41aa-b915-2f390ab508e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:48.744240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8aa9bd-dd48-41aa-b915-2f390ab508e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:48.744955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8aa9bd-dd48-41aa-b915-2f390ab508e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:48.746394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8aa9bd-dd48-41aa-b915-2f390ab508e3 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:48.774619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 597/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:13:48 2022] POST /identity/v3/projects => generated 361 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:13:50.779977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c738e-1f4d-4fb2-9539-fce5278a6d12 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:50.780416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c738e-1f4d-4fb2-9539-fce5278a6d12 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:50.780557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c738e-1f4d-4fb2-9539-fce5278a6d12 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:50.781933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 577/1174] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:50.787247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:50.787650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:50.787807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:50.804391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:50.805569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:50.822282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:50.872034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 598/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:50.897120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c738e-1f4d-4fb2-9539-fce5278a6d12 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:50.998938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c738e-1f4d-4fb2-9539-fce5278a6d12 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:51.031986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b31eacb-1524-4006-8789-34f0c3c415d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:51.032857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31eacb-1524-4006-8789-34f0c3c415d4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:51.033147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31eacb-1524-4006-8789-34f0c3c415d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:51.033426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31eacb-1524-4006-8789-34f0c3c415d4 None admin] PATH_INFO: `/v3/domains/5201235c0684489ea6d2258db1c1c15f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:51.034592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31eacb-1524-4006-8789-34f0c3c415d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5201235c0684489ea6d2258db1c1c15f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:51.036023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b31eacb-1524-4006-8789-34f0c3c415d4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:51.037666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 578/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Fri Jul 15 19:13:50 2022] GET /identity/v3/domains/5201235c0684489ea6d2258db1c1c15f => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:13:51.043448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:51.137982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb96172-07dc-440a-9e34-60917dd02f07 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:51.169920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1eda4e-7e03-4d68-8115-a3aca890f4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:51.170581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eda4e-7e03-4d68-8115-a3aca890f4d4 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:51.170860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eda4e-7e03-4d68-8115-a3aca890f4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:51.171131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1eda4e-7e03-4d68-8115-a3aca890f4d4 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:51.171833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1eda4e-7e03-4d68-8115-a3aca890f4d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:51.173298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1eda4e-7e03-4d68-8115-a3aca890f4d4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:51.194374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 599/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:13:51 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:13:53.224467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0df18-6f16-4936-bc1f-ca35499029f4 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:53.225024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0df18-6f16-4936-bc1f-ca35499029f4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:53.225024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0df18-6f16-4936-bc1f-ca35499029f4 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:53.226376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 579/1178] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:53.231476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:53.231880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:53.232025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:53.249832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:53.251007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:53.253140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:53.317284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 600/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:53.340397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0df18-6f16-4936-bc1f-ca35499029f4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:53.365010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:53.440930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0df18-6f16-4936-bc1f-ca35499029f4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:53.476410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:53.477113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:53.477397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:53.477675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:53.488626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:53.496099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:53.497511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:53.504126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 580/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:53.505036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:53.598385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efed2-aa87-49cf-af96-9fe023695401 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:53.631185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:53.631833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:53.632130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:53.632403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:53.633136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:53.633437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:53.634796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:53.639564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:53.643418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 601/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:53.648680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:53.727857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:53.746253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5e954-be8b-4c38-b5db-99ccec6936d8 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:53.778919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:53.779787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:53.780066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:53.780374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] PATH_INFO: `/v3/users/bcb6a8619078443daa70edea69cfcccb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:53.795068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] RBAC: Authorizing `identity:get_user(user_id=bcb6a8619078443daa70edea69cfcccb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:53.796552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:53.798235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 581/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:53 2022] GET /identity/v3/users/bcb6a8619078443daa70edea69cfcccb => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:53.804880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:53.901523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee87a66c-85ad-4009-8d32-9c52d53ddc97 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:53.934190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:53.934831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:53.935114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:53.935391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:53.998078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:54.000344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:54.002812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:54.097738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 602/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:13:54.103636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:54.199054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71f528-8f83-4172-b7cb-58ac651ba321 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:54.232162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a41d93-a986-4dfa-9667-668fb1a0c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:54.232994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a41d93-a986-4dfa-9667-668fb1a0c3ff None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:54.233271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a41d93-a986-4dfa-9667-668fb1a0c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:54.233539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a41d93-a986-4dfa-9667-668fb1a0c3ff None admin] PATH_INFO: `/v3/projects/0496f9112fde48ab89ba9d799745a126` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:54.234691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a41d93-a986-4dfa-9667-668fb1a0c3ff None admin] RBAC: Authorizing `identity:get_project(project_id=0496f9112fde48ab89ba9d799745a126)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:54.236160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a41d93-a986-4dfa-9667-668fb1a0c3ff None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:54.237975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 582/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:54 2022] GET /identity/v3/projects/0496f9112fde48ab89ba9d799745a126 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:54.244139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:54.336964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:54.339914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23452ad0-1c95-45a3-8492-a43c5137894b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:54.373200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:54.373832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:54.374112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:54.374385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:54.375200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=0496f9112fde48ab89ba9d799745a126, user.id=bcb6a8619078443daa70edea69cfcccb) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:54.375506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:54.377105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34282862-5c34-4586-8b52-9625f8f35d03 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:54.382379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 603/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:13:54 2022] GET /identity/v3/role_assignments?user.id=bcb6a8619078443daa70edea69cfcccb&scope.project.id=0496f9112fde48ab89ba9d799745a126&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:56.350558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e257d-b715-44b3-b3cf-5594c2a3157e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:56.351092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e257d-b715-44b3-b3cf-5594c2a3157e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:56.351092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e257d-b715-44b3-b3cf-5594c2a3157e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:56.352508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 583/1186] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:56.357744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:56.358181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:56.358283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:56.375399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:56.377266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:56.441261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 604/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:56.462767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e257d-b715-44b3-b3cf-5594c2a3157e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:56.473319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:56.562482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e257d-b715-44b3-b3cf-5594c2a3157e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:56.595177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:56.595895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:56.596201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:56.596476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:56.606156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:56.614258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:56.615384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:13:56.616326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 584/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:56.621717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:56.713874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:56.721897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3984b5ad-38b2-40d2-bc40-6957fd760db2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:56.756313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:56.756981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:56.757265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:56.757543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:56.758292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:56.758602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:56.759981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:56.768595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 605/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:56.774259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:56.839231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:56.873177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52086b29-b510-49bd-a118-7ec729521f2b None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:56.905685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:56.906537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:56.906820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:56.907098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] PATH_INFO: `/v3/users/bcb6a8619078443daa70edea69cfcccb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:56.909119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] RBAC: Authorizing `identity:get_user(user_id=bcb6a8619078443daa70edea69cfcccb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:56.910546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:56.912192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 585/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:56 2022] GET /identity/v3/users/bcb6a8619078443daa70edea69cfcccb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:56.917904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:57.015086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c92eaa-1a00-4411-9081-265d667728b4 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:57.047218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289bf4e1-8226-4685-a252-b9af2795d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:57.048068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bf4e1-8226-4685-a252-b9af2795d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:57.048934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bf4e1-8226-4685-a252-b9af2795d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:57.049208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289bf4e1-8226-4685-a252-b9af2795d4a0 None admin] PATH_INFO: `/v3/projects/0496f9112fde48ab89ba9d799745a126` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:57.050369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bf4e1-8226-4685-a252-b9af2795d4a0 None admin] RBAC: Authorizing `identity:get_project(project_id=0496f9112fde48ab89ba9d799745a126)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:57.051799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289bf4e1-8226-4685-a252-b9af2795d4a0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:57.053492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 606/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:13:56 2022] GET /identity/v3/projects/0496f9112fde48ab89ba9d799745a126 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:57.059432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:57.159941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c49f9-5061-4d6d-b88f-5eb2593814aa None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:57.192906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca8e60b-91c8-4545-beec-3808af0487a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:57.193792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8e60b-91c8-4545-beec-3808af0487a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:57.194076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8e60b-91c8-4545-beec-3808af0487a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:57.194356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca8e60b-91c8-4545-beec-3808af0487a7 None admin] PATH_INFO: `/v3/projects/0496f9112fde48ab89ba9d799745a126/users/bcb6a8619078443daa70edea69cfcccb/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:57.196332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8e60b-91c8-4545-beec-3808af0487a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0496f9112fde48ab89ba9d799745a126, user_id=bcb6a8619078443daa70edea69cfcccb, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:57.197764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca8e60b-91c8-4545-beec-3808af0487a7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:57.200776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:57.212320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 586/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:13:57 2022] PUT /identity/v3/projects/0496f9112fde48ab89ba9d799745a126/users/bcb6a8619078443daa70edea69cfcccb/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:13:59.182241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11681-7d5d-4dd0-ab13-14866725621f None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.182786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11681-7d5d-4dd0-ab13-14866725621f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.182879 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11681-7d5d-4dd0-ab13-14866725621f None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.184213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 607/1193] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:13:59.190154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.190637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.190753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.207724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:13:59.208943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:13:59.273687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 587/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:13:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:13:59.296649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11681-7d5d-4dd0-ab13-14866725621f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.323130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.397641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11681-7d5d-4dd0-ab13-14866725621f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.429891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.430672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.430951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.431224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.442099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:13:59.449481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:13:59.450569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:13:59.451431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 608/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:13:59.456849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.551349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e419255-6070-4542-a479-e4cf0af97b92 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.576798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.585677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.586330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.586610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.586885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.587626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:13:59.587928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.589310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:59.597406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 588/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:59.603292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.685564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.702147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bb7e24-deaf-45a9-8fdd-8c424aeff776 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.734604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.735990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.736325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.736600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] PATH_INFO: `/v3/users/bcb6a8619078443daa70edea69cfcccb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.737770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] RBAC: Authorizing `identity:get_user(user_id=bcb6a8619078443daa70edea69cfcccb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.739204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:13:59.740876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 609/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/users/bcb6a8619078443daa70edea69cfcccb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:13:59.746784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:13:59.843364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb71f3-0d37-4a86-ac4e-352984f31dfd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:13:59.876568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:13:59.877209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:13:59.877488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:13:59.877759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:13:59.930105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:13:59.943184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:13:59.944655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:00.039386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 589/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:13:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:14:00.045185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:00.143368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7f62f-9d7d-4fb8-9835-e8a271f38d2b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:00.175743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a67f02-416b-4abb-ad34-a89afbe4ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:00.176617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a67f02-416b-4abb-ad34-a89afbe4ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:00.176900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a67f02-416b-4abb-ad34-a89afbe4ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:00.177175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a67f02-416b-4abb-ad34-a89afbe4ceb4 None admin] PATH_INFO: `/v3/projects/0496f9112fde48ab89ba9d799745a126` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:00.178322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a67f02-416b-4abb-ad34-a89afbe4ceb4 None admin] RBAC: Authorizing `identity:get_project(project_id=0496f9112fde48ab89ba9d799745a126)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:00.179742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a67f02-416b-4abb-ad34-a89afbe4ceb4 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:00.181430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 610/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Fri Jul 15 19:14:00 2022] GET /identity/v3/projects/0496f9112fde48ab89ba9d799745a126 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:00.187564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:00.275404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:00.288695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79eea3-c73f-4375-be59-8e70f1b526a3 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:00.322336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:00.322928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:00.323214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:00.323495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:00.324349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=0496f9112fde48ab89ba9d799745a126, user.id=bcb6a8619078443daa70edea69cfcccb) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:00.324666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:00.326096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9d2c60-a997-4d46-b520-79d96b198f0f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:00.333014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 590/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:14:00 2022] GET /identity/v3/role_assignments?user.id=bcb6a8619078443daa70edea69cfcccb&scope.project.id=0496f9112fde48ab89ba9d799745a126&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:10.171682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:10.329293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=24 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:55.494363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e77f57-855f-4582-bd61-46f852ac02a5 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.495221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e77f57-855f-4582-bd61-46f852ac02a5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.495221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e77f57-855f-4582-bd61-46f852ac02a5 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.497030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 611/1201] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:55.503528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.504031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.504290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.523136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:55.524354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:55.621476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 591/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:14:55.645379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e77f57-855f-4582-bd61-46f852ac02a5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:55.784742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e77f57-855f-4582-bd61-46f852ac02a5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:55.836814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:55.837556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:55.837842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:55.838122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:55.851968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:55.853691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:14:55.877314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:55.878458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:55.879416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:14:55.879416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:14:55.879416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:14:55.879416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:14:55.883715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 612/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:55.885436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.030155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac230-d887-4d39-83af-9873aa61cba2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.071297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:56.079467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.080431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.080722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.080999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.081817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:56.082031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:56.083766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9581386-a55a-42e3-8e4d-ca21e14b0d11 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:56.105454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 592/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:56.107195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:56.183630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:56.263322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb4791-b97f-4e53-ba19-173235bde0a7 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:56.302143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03aa9c2-c529-4b48-bf0b-9a82a7ac0887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:56.302938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03aa9c2-c529-4b48-bf0b-9a82a7ac0887 None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:56.303335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03aa9c2-c529-4b48-bf0b-9a82a7ac0887 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:56.304296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03aa9c2-c529-4b48-bf0b-9a82a7ac0887 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:56.304556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03aa9c2-c529-4b48-bf0b-9a82a7ac0887 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:56.306180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03aa9c2-c529-4b48-bf0b-9a82a7ac0887 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:56.334088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 613/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Fri Jul 15 19:14:56 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:14:58.542362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ba6ff-e5e4-4fff-8b47-51ac3765b627 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.542996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ba6ff-e5e4-4fff-8b47-51ac3765b627 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.542996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ba6ff-e5e4-4fff-8b47-51ac3765b627 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.544464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 593/1206] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:14:58.553033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.553604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.553701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.571894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:14:58.573236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:14:58.648168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 614/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:14:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:14:58.672529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ba6ff-e5e4-4fff-8b47-51ac3765b627 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.798100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ba6ff-e5e4-4fff-8b47-51ac3765b627 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:58.844808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:58.845685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:58.845922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:58.846214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:58.858376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:14:58.868988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:58.870253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:14:58.871110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:14:58.879175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 594/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:58.879352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:58.922368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:58.990277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3de332-ce49-467f-aa9a-167cf4a646a1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.030856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.031626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.031906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.032232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.032969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:59.033277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.034672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.049794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 615/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:14:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:59.055776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.111304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.166979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa61d7f-9946-4b92-9c5a-36b7c364d5a2 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.204376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.205233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.205456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.205772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.216259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.218410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:14:59.237429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:59.238563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:59.240009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:14:59.240009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:14:59.240009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:14:59.240009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:14:59.240009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 595/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:59.245858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.357814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.361333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3ec03-8760-4269-82b7-532f2244a5b0 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.399995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.400698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.400970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.401273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.401976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:59.402282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.403662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.425056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 616/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:59.426368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.541404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30258ca-90a7-41ce-9611-45161a9d76d4 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.543054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:14:59.580338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.581431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.581704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.581983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.594889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.596386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:14:59.612328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:14:59.613382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:14:59.614369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:14:59.619952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 596/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:14:59.620117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.736562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72b14a-0bec-4e24-9d3f-c11b859afe72 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.782154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c193883-c471-49a3-9535-328747c22edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.783479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193883-c471-49a3-9535-328747c22edb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.783479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193883-c471-49a3-9535-328747c22edb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.783479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c193883-c471-49a3-9535-328747c22edb None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:14:59.784294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c193883-c471-49a3-9535-328747c22edb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:14:59.784606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c193883-c471-49a3-9535-328747c22edb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:14:59.786107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c193883-c471-49a3-9535-328747c22edb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:14:59.805486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 617/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:14:59.811564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:14:59.929329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd20e7a-cfec-468a-85f8-865f6363b96c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:14:59.976600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:14:59.977754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:14:59.977754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:14:59.977948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.012757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.059459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.061383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.178654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 597/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:14:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:15:00.184697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c193883-c471-49a3-9535-328747c22edb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.205257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.296311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c193883-c471-49a3-9535-328747c22edb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.336975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.338532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.338532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.338532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.349648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.351230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:00.369353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:00.370583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:00.371479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:00.371479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:00.371479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:00.371479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:00.376331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 618/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:00.377448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.451408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.496876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6015c3-c833-428f-81ae-a864b2f1e90e None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.534994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.535801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.536219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.536595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.537419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:00.537835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.539348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.552827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 598/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:00.558136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.636579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.672213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea2897-f182-41e4-848f-b90e82b57b9e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.719362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.719362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.719362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.719362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.726957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.728655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:00.747006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:00.748630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:00.749520 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:00.752247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 619/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:00.754693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:00.869447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf873ef4-2513-41b2-883c-2041cbbae668 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:00.894115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:00.907351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:00.908016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:00.908330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:00.908604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:00.909393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:00.909720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:00.911130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a4f2c-b206-4b28-a6cf-30c6e5ca8059 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:00.927479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 599/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:00.931991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:01.039072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c15b7-d6d6-46d3-8803-0e2188fa3788 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:01.068534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:01.074934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:01.075647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:01.075981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:01.076663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:01.077231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=350d4be3651647829ae3f7e903326891) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:01.077556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:01.078981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3892b3c-6b77-46a1-ba98-27f205d93e0b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:01.096547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 620/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:00 2022] GET /identity/v3/role_assignments?user.id=350d4be3651647829ae3f7e903326891&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:03.188204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71fc61-d69f-4c01-85c1-a5f1c81a1c49 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:03.188762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71fc61-d69f-4c01-85c1-a5f1c81a1c49 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:03.188762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71fc61-d69f-4c01-85c1-a5f1c81a1c49 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:03.190247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 600/1220] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:03.195968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:03.196444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:03.196569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:03.215132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:03.216430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:03.293672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 621/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:03.319982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71fc61-d69f-4c01-85c1-a5f1c81a1c49 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:03.434596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71fc61-d69f-4c01-85c1-a5f1c81a1c49 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:03.476395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:03.477127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:03.477410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:03.477690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:03.490258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:03.499657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:03.500795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:03.501832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 601/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:03.507467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:03.622136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86c8f95-2fb4-41b8-8db1-20765400b187 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:03.647850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:03.661455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:03.662164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:03.662447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:03.662723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:03.663563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] RBAC: Adding query filter params (name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:03.663925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:03.665414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:03.674764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 622/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:03.680858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:03.735618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:03.815600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fac5e-486f-462e-b868-f7480dd4a9b0 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:03.850375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:03.851167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:03.851449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:03.851723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:03.862807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:03.864280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:03.881787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:03.882914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:03.884179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:03.884179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:03.884179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:03.884179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:03.884179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 602/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:03.889427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:03.995948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfdcaa-f30d-4e52-9629-26bbedc7301e None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:04.029818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:04.030484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.030768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.031044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.032012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:04.032416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:04.033822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:04.046280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 623/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:04.053549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:04.163433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:04.172522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07066f28-c76d-4345-a2f6-197d59e402d6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:04.207652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:04.208504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.212127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.212127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.217338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:04.218790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:04.235685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:04.236797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:04.244300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 603/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:04.245282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:04.298598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:04.360174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826cc3d7-f6cb-48d4-b935-7d5ad75a7f1c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:04.396250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:04.396925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.397209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.397488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.398320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:04.398628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:04.400066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:04.419003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 624/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Jul 15 19:15:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:04.425754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:04.535592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9fd5-a0c3-45a9-8c05-cb7c52e0abd5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:04.571650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:04.572614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.572955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.573311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.585094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:04.586557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:04.604795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:04.605887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:04.607491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:04.607491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:04.607491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:04.607491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:04.607491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 604/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:04.613266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:04.716035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13a06d-01ef-4409-99c3-d7f82734e089 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:04.719333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:04.757862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:04.758533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.758817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.759101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.759830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:04.760164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:04.761553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:04.780675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 625/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:04.782049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:04.792675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:04.899555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bc0e9-aa60-472c-ade5-d9d50abec50a None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:04.938950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:04.939720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:04.939996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:04.940294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:04.947166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:04.948674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:04.962757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:04.964003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:04.965532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 605/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:04.970283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.081197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255caf-445c-4711-8062-ae625bb1f4bc None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.118519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.119179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.119459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.119985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.120832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:05.121119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.122579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8612e16-7910-4890-8f76-ba560674df63 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.135961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 626/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:05.142559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:05.213434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:05.252230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae77ee7-2854-4ac7-88de-da2bc2869d72 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:05.289065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865fd3a-f4b0-4b2a-a74e-b60082acc10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:05.289962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865fd3a-f4b0-4b2a-a74e-b60082acc10f None admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:05.290244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865fd3a-f4b0-4b2a-a74e-b60082acc10f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:05.290518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865fd3a-f4b0-4b2a-a74e-b60082acc10f None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/350d4be3651647829ae3f7e903326891/roles/a37f5fd9b4904f06bf2168d25dce5506` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:05.306840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865fd3a-f4b0-4b2a-a74e-b60082acc10f None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=350d4be3651647829ae3f7e903326891, role_id=a37f5fd9b4904f06bf2168d25dce5506)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:05.308741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865fd3a-f4b0-4b2a-a74e-b60082acc10f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:05.323456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 606/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:05 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/350d4be3651647829ae3f7e903326891/roles/a37f5fd9b4904f06bf2168d25dce5506 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:07.517907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a9e94-9351-4473-b496-f194576ea734 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:07.518463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a9e94-9351-4473-b496-f194576ea734 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:07.518463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a9e94-9351-4473-b496-f194576ea734 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:07.519832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 627/1233] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:07.525481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:07.525936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:07.526041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:07.546029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:07.547240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:07.620304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 607/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:07.648340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a9e94-9351-4473-b496-f194576ea734 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:07.674217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:07.756598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a9e94-9351-4473-b496-f194576ea734 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:07.798942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722957fd-092c-429f-915c-301a787b5667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:07.799742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722957fd-092c-429f-915c-301a787b5667 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:07.800049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722957fd-092c-429f-915c-301a787b5667 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:07.800354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722957fd-092c-429f-915c-301a787b5667 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:07.811377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722957fd-092c-429f-915c-301a787b5667 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-722957fd-092c-429f-915c-301a787b5667 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:07.819376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:07.820519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 15 19:15:07.821372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:07.826374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 628/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Fri Jul 15 19:15:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:07.827611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:07.889926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:07.947583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc1e18-6bf8-4987-9057-2787b2eddad2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:07.983203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:07.983900 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:07.984218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:07.984522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:07.985291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:07.985644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:07.987015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:07.996015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 608/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Fri Jul 15 19:15:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:08.002927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722957fd-092c-429f-915c-301a787b5667 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.091627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:08.110477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722957fd-092c-429f-915c-301a787b5667 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:08.145319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:08.146254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.146563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.146812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.158807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:08.160303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:08.178255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:08.179330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:08.180598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:08.180598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:08.180598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:08.180598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:08.180598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 629/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:08.185987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.294607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa4b1f0-46f0-4255-b937-dd80b6d39ad6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:08.305905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:08.335024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:08.335726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.336011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.336319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.337039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:08.337344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:08.338740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:08.351273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 609/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:08.357356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.475580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a3fca-8d42-4fc9-a048-6e7695b5a131 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:08.522660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:08.523509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.523795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.524075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.526942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:08.533229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:08.535111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:08.550939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:08.552859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:08.554755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 630/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:08.558773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.672310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4257f6-cd72-4e17-bb52-a2310091d68d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:08.708495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:08.709168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.709453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.709731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.710566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:08.710877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:08.712460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:08.730549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 610/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:08.737075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:08.851749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4962a-c67f-474d-9a8e-8947cf12578a None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:08.893266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:08.894091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:08.894578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:08.894990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:08.962958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:08.970208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:08.971764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.090680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 631/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 15 19:15:09.098727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.142700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.235888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a5804-1f2e-490d-8240-f774588e8a9c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.275573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.276444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.276840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.277208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.297073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.299894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:09.332766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.334764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.335868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:09.335868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:09.335868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:09.335868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:09.335868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 611/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:09.344545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.462866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.470784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b72f5-ba8f-4e51-b7f1-b85b36500a1b None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.519125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.519826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.520206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.520561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.521444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.521878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.523539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.538287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 632/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.545237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.633437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:09.657574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f8d62-79af-4440-845d-9f480eaf2dbd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.694033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.694824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.695107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.695381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.702158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.703607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:09.720927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:09.722044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:09.723286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 612/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:09.728361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:09.851656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c20b44-f079-400c-b440-8317210e8c28 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:09.892707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:09.893395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:09.893681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:09.893961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:09.894763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:09.895076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:09.896577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d9c1a-cce9-41d4-a786-12304dd93f00 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:09.910124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 633/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:09.916433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:10.035787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603ec7a9-c083-4457-9226-bc53c5c45268 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:10.073272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:10.073934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:10.074245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:10.074706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:10.075948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] RBAC: Adding query filter params (role.id=a37f5fd9b4904f06bf2168d25dce5506, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=350d4be3651647829ae3f7e903326891) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:10.076294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:10.078150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0b66c-5e51-422c-8e70-0dcf334c0be2 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:10.087703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 613/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:09 2022] GET /identity/v3/role_assignments?user.id=350d4be3651647829ae3f7e903326891&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=a37f5fd9b4904f06bf2168d25dce5506 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.188545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a2db5-da8d-430c-aefe-10a20b26f687 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.189135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a2db5-da8d-430c-aefe-10a20b26f687 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.189135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a2db5-da8d-430c-aefe-10a20b26f687 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.190595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 634/1247] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:15:12.196057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.196655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.196770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.203063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.216673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:12.217980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:12.294396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 614/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:12.321530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a2db5-da8d-430c-aefe-10a20b26f687 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.342383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.437940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a2db5-da8d-430c-aefe-10a20b26f687 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.480236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.480924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.481226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.482334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.494573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:12.505456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:12.506615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:12.507620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 635/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:12.513256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.626264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab061084-9ab4-43af-bc60-954e89455527 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.661825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.667669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.668380 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.668662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.669485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.670240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:12.670546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.672130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.682146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 615/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:12.689605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:12.788454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:12.816253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96cfc01-b961-4f2c-b49e-b25d00f5685d None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:12.857084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:12.858200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:12.858489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:12.858880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:12.873710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:12.875211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:12.895138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:12.896356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:12.897611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:12.897611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:12.897611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:12.897611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:12.897611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 636/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:12.903397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.022256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99c44e-15a1-4946-bb49-5e66871391ec None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.060414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.060927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.061213 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.061587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.062327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.062640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.064513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.078003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 616/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.083710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.200591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ad5a0-ba98-4354-aebe-a1d9b6388c2c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.230932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.240285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.241215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.241546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.241854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.250654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.252216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:13.269081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:13.270248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:13.271713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 637/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:13.277136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.343906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.398798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6186dca-470e-4465-a26e-c7659ff19f07 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.441435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.442449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.442813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.443128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.444294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:13.444607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.446274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.470105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 617/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:13.477471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.601354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13040c1-259b-4515-a6ab-f4cc602771fb None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.642168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:13.643035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:13.643317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:13.643590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:13.708317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:13.722809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:13.724577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:13.832115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 638/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:15:13.837344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:13.960163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc3aec-e44a-487a-832e-32bfcd9a8f70 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:13.976425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.000502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.001413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.001658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.001934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.014987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.016964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:14.039171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:14.040625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:14.042133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:14.042133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:14.042133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:14.042133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:14.042133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 618/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:14.046908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.139068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.154314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5338f-4f72-4f9a-ad0b-dced759dfe03 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.191366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.192040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.192365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.192646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.193376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:14.193690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.195069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.207332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 639/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:14.213992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.331488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e716dcb-51a3-478c-b553-3864ec7e8a5c None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.374131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.375011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.375360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.375701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.383992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.385640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:14.400534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:14.401762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:14.402982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:14.407242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 619/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 15 19:15:14.408249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.530100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e046d7d-57f9-4eb4-af42-2b3d4abb25b9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.567434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.568212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.568501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.568787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.569578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:14.569927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.571448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ab6f5-2ea2-4209-bf66-710c77590caf None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.585713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 640/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:14.592016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:14.620540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:14.705500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b4115-8494-4e6c-af58-8ed2aca36884 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:14.744789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:14.745290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:14.745583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:14.746870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:14.747042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=350d4be3651647829ae3f7e903326891) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:14.747572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:14.749218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d75fb-20cb-4b2f-9467-f6375bed2efe None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:14.755807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 620/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:14 2022] GET /identity/v3/role_assignments?user.id=350d4be3651647829ae3f7e903326891&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=56597e05853f4660b09a96c651dd0c7a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:16.810809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae818f-b0e8-4a2f-82a2-c27953c11d6e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.811247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae818f-b0e8-4a2f-82a2-c27953c11d6e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.811439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae818f-b0e8-4a2f-82a2-c27953c11d6e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.812733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 641/1261] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:16.817828 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:16.818195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:16.818389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:16.835340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:16.836550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:16.902101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 621/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:16.924867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae818f-b0e8-4a2f-82a2-c27953c11d6e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:16.935487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.024150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae818f-b0e8-4a2f-82a2-c27953c11d6e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.057377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.058153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.058433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.058705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.068594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:17.076310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.077391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:17.078436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 642/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.083438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.101894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.186077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129aaa1-6768-4801-94d4-e1f9885cd4ee None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.220070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.220756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.221039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.221313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.222059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.222365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.223735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.232310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 622/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.238030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.302567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.334593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4ac9e1-e6e5-4a0f-b49c-ecf9f6d7e345 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.367659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.368458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.368739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.369009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.379500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.381058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:17.398045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.399109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.400006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:17.400006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:17.400006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:17.400006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:17.400006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 643/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.406085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.497750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.522157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411328-5ec7-47fc-a913-ed1c4bad5762 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.555565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.556179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.556465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.556739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.557458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.557764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.559738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.571202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 623/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.576953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.677068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f11a653-bcdb-45f4-815f-1dac65d7518c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.696913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.711325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.712139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.712423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.712695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.720334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.721774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:17.736069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:17.737132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:17.738097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 644/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:17.743045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:17.847899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b08dc-9d0e-47a7-9170-4e1bf90fc7a5 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:17.881976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:17.882639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:17.882926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:17.883206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:17.884032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:17.884374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:17.885791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:17.894662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:17.904794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 624/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:17.910314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.006321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5209bc-57aa-4da9-84e0-34078725a253 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.038333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.039100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.039398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.039651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.050423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.051848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:18.068330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.069342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.070455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:18.070455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:18.070455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:18.070455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:18.072234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 645/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.075169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.172182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b9f81-eb59-4549-9582-e20a0a15e08d None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.205001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.205645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.205924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.206196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.206911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.207212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.208622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.220002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 625/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:18.225761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.257988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.323796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643915f2-dfc1-4fe2-bac3-48fc62956de9 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.355255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.356579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.356860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.357209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.363577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.365038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:18.379510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:18.380541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:18.381509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 646/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:18.386384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.444261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.494989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd25c2-9f95-481f-a111-14465e231285 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.531861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.532548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.532835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.533114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.533899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:18.534207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.535647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d4ed7-0d78-4a97-8888-a9bae9778e94 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.548221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 626/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:15:18.554437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:18.644347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:18.653986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a1bb3-7bb3-4d32-96de-5410f53d4551 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:18.686917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c11e4d3-ac39-4ae3-b7e1-26c97bdb514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:18.687794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11e4d3-ac39-4ae3-b7e1-26c97bdb514c None admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:18.688105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11e4d3-ac39-4ae3-b7e1-26c97bdb514c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:18.688384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c11e4d3-ac39-4ae3-b7e1-26c97bdb514c None admin] PATH_INFO: `/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/350d4be3651647829ae3f7e903326891/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:18.690329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11e4d3-ac39-4ae3-b7e1-26c97bdb514c None admin] RBAC: Authorizing `identity:create_grant(project_id=d476a68e6b254166ae1df0ccdc93e83d, user_id=350d4be3651647829ae3f7e903326891, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:18.691763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c11e4d3-ac39-4ae3-b7e1-26c97bdb514c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:18.703904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 647/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Fri Jul 15 19:15:18 2022] PUT /identity/v3/projects/d476a68e6b254166ae1df0ccdc93e83d/users/350d4be3651647829ae3f7e903326891/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:15:20.677789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688dd1d-d3cf-44d4-b59b-c043e5505667 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.678361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688dd1d-d3cf-44d4-b59b-c043e5505667 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.678361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e688dd1d-d3cf-44d4-b59b-c043e5505667 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.679716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 627/1274] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:20.685363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.685803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.685912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.702081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:20.703252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:20.770548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 648/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:20.793215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688dd1d-d3cf-44d4-b59b-c043e5505667 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:20.892888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e688dd1d-d3cf-44d4-b59b-c043e5505667 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:20.910101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:20.927062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:20.927887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:20.928143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:20.928423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:20.938266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 15 19:15:20.945834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:20.946951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:20.947994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 628/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:20.953101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.030604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.048932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea65f45-5704-4b8b-8095-5a09969805d7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.080826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.081481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.081762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.082035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.082766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] RBAC: Adding query filter params (name=admin) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:21.083068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.084459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.092739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 649/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Fri Jul 15 19:15:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:21.098315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.194760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63071a9-b0e1-43f9-8d72-542712c83528 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.226897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.227676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.227962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.228268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.238924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.240381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:21.257191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:21.258225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:21.259254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:21.259254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:21.259254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:21.259254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:21.259254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 629/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:21.264623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.361509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9967a-fe37-46f4-bde7-802addb273af None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.394437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.395090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.395374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.395648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.396389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:21.396694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.398057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.412248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 650/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:21.417975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.448004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.517716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a2b85b-d6bd-4bdf-9e04-05368b910ad9 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.550588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.551365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.551646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.551920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.560158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.561593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 15 19:15:21.576119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:21.577235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:21.578167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 630/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:21.583270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.587060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.685125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93861b14-5783-4646-9652-00828a10ae72 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.718365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.718951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.719236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.719510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.720368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:21.720677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.722102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:21.739307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 651/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:21.745432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:21.837283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:21.847528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c972de4-7932-4cba-a022-a77ad61c17dd None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:21.879937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:21.880607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:21.880889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:21.881162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:21.945638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:21.947087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.033354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.044013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 631/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Fri Jul 15 19:15:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 15 19:15:22.049811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.109623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.148487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8931-a320-4a5e-b5b7-9672631875e5 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.185873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.186654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.186935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.187209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.196906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.198360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 15 19:15:22.215410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:22.216410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:22.217517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 15 19:15:22.217517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 15 19:15:22.217517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 15 19:15:22.217517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:22.217517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 652/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:22.222467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.319480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4f9c18-0faa-42b6-9945-e4afaf5f16a6 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.352304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.352961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.353242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.353517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.354240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:22.354548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.356614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.367764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 632/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:22.374193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.470435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d31109-9bc8-4818-a9e8-5f76733d406c None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.484536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.504519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.505300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.505582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.505859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.512484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.513926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 15 19:15:22.527366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:22.529048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: ERROR keystone.server.flask.application Jul 15 19:15:22.529948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 653/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:22.534828 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.583579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:22.633697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16d3ca-ea0e-48c6-8435-0547e600e25f None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.666345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.666990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.667271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.667543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.668366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:22.668834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.670251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d38e3-d5cf-4f8d-8ae5-ab0ae319544b None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.683534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 633/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:22.689693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:22.789629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501a188-97c6-4dd5-b4ce-75801c63b470 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:22.821206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:22.821846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:22.822128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:22.822402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:22.823213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] RBAC: Adding query filter params (role.id=56597e05853f4660b09a96c651dd0c7a, scope.project.id=d476a68e6b254166ae1df0ccdc93e83d, user.id=350d4be3651647829ae3f7e903326891) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:22.823519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:22.825531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc9d5c-3994-4ac1-8d02-e9ef1fa9f850 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:22.831171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 654/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Fri Jul 15 19:15:22 2022] GET /identity/v3/role_assignments?user.id=350d4be3651647829ae3f7e903326891&scope.project.id=d476a68e6b254166ae1df0ccdc93e83d&role.id=56597e05853f4660b09a96c651dd0c7a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:24.841131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fe10c-5c28-4c97-85a3-8439b2c498e8 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.841654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fe10c-5c28-4c97-85a3-8439b2c498e8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.841654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fe10c-5c28-4c97-85a3-8439b2c498e8 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.843043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 634/1288] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:24.848255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:24.848676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:24.848817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:24.865157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:24.866334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:24.895778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:24.931902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 655/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:24.954396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fe10c-5c28-4c97-85a3-8439b2c498e8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.056451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fe10c-5c28-4c97-85a3-8439b2c498e8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.067579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:25.092435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.093217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.093499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.093774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.094420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.095785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:25.107535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:25.108772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:25.109956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 635/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:25.114763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.213558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ddd3f-afa3-4e3d-90af-525c540c9fc2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.247856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.248537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.248825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.249104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.249807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.250117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.251483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c71007-20aa-42ab-a79b-3c3ca4b8c83f None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.256210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 656/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:25.262317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:25.357453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0c93b6-8634-46b4-9c13-8db2cf2fee96 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:25.390405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:25.391039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:25.391319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:25.391595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] PATH_INFO: `/v3/services` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:25.392317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:25.392623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:25.393978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999ce1-25d3-43e8-9612-69ea7305cdf7 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:25.398261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 636/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Jul 15 19:15:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:27.441024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6f19-fa16-4104-a28a-09f46a3da929 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.441638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6f19-fa16-4104-a28a-09f46a3da929 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.441858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6f19-fa16-4104-a28a-09f46a3da929 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.443463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 657/1293] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:27.451359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2414-9693-4732-a218-5cc28f30d43a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.451860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2414-9693-4732-a218-5cc28f30d43a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.452356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb2414-9693-4732-a218-5cc28f30d43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.473636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-90eb2414-9693-4732-a218-5cc28f30d43a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:27.475136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-90eb2414-9693-4732-a218-5cc28f30d43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:27.516666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.569688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 637/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:27.592288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6f19-fa16-4104-a28a-09f46a3da929 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:27.625064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:27.707919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6f19-fa16-4104-a28a-09f46a3da929 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:27.751896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e09fba-03c6-410f-abf0-14cdded70f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:27.752602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e09fba-03c6-410f-abf0-14cdded70f8d None admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:27.752885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e09fba-03c6-410f-abf0-14cdded70f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:27.753188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e09fba-03c6-410f-abf0-14cdded70f8d None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:27.753910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e09fba-03c6-410f-abf0-14cdded70f8d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:27.755288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e09fba-03c6-410f-abf0-14cdded70f8d None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:27.773043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 658/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Fri Jul 15 19:15:27 2022] POST /identity/v3/services => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:29.790858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220384-39c8-4ea1-90fa-6464c1d7b3c2 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.791318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220384-39c8-4ea1-90fa-6464c1d7b3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.791415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07220384-39c8-4ea1-90fa-6464c1d7b3c2 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.792887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 638/1296] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:29.797963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:29.798378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:29.798526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:29.815315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:29.816536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:29.888182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 659/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:29.910190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220384-39c8-4ea1-90fa-6464c1d7b3c2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.009227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07220384-39c8-4ea1-90fa-6464c1d7b3c2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.041532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.042242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.042522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.042832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.043478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.044874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:30.056203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:30.057342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:30.058917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 639/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:30.063771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.082232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.161565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d3beaa-a57d-4362-adce-c69594b39ce7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.195196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.195854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.196168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.196448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.197154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:30.197463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.198845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.204123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 660/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:30.210174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.241435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.308111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e6d33d-06b5-4caa-8977-2952382c2881 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.340514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.341153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.341433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.341707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.342485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] RBAC: Adding query filter params (interface=public, service_id=a3a19dc553bb46ab9040ae037ef5ecb4, region_id=RegionOne) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:30.342790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.344162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72ac8-82e4-4aaf-87ca-9cb9a5f59fff None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.349051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 640/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/endpoints?service_id=a3a19dc553bb46ab9040ae037ef5ecb4&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:30.354609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:30.437278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:30.459862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c52a4a8-975b-474d-b0aa-f3dfa96f7d62 None admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:30.501941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5e873-8239-4bbc-b440-f2c4e8964e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:30.502733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5e873-8239-4bbc-b440-f2c4e8964e28 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:30.503090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5e873-8239-4bbc-b440-f2c4e8964e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:30.503440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5e873-8239-4bbc-b440-f2c4e8964e28 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:30.504151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5e873-8239-4bbc-b440-f2c4e8964e28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:30.505537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5e873-8239-4bbc-b440-f2c4e8964e28 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:30.512444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 661/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Fri Jul 15 19:15:30 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:15:32.498893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b1c1-91e0-4217-a15d-57a60610ec46 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.499453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b1c1-91e0-4217-a15d-57a60610ec46 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.499453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81b1c1-91e0-4217-a15d-57a60610ec46 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.500862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 641/1302] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:15:32.505969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.506401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.506527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.522924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:15:32.524132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:15:32.589964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 662/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:15:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 15 19:15:32.612468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b1c1-91e0-4217-a15d-57a60610ec46 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:32.704359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:32.711243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81b1c1-91e0-4217-a15d-57a60610ec46 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:32.743543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:32.744640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.744918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.745197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.746457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:32.749061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 15 19:15:32.763529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 15 19:15:32.764807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.get_or_create( Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application with Lock( Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self._enter() Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application return self.creator() Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application created_value = creator( Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: ERROR keystone.server.flask.application Jul 15 19:15:32.765989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 642/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Fri Jul 15 19:15:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 15 19:15:32.770850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:32.845317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:32.867854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac9d9ad-90d0-472b-8581-f80a83c05b88 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:32.900678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:32.901333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:32.901614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:32.901888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] PATH_INFO: `/v3/services` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:32.903256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:15:32.903560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:32.904963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42109c61-2882-4ecd-a0af-767d84d49f02 None admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:32.910130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 663/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Fri Jul 15 19:15:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:15:32.915910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:15:33.013285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878395d9-391e-495c-85bd-c650ca1e7a09 None admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:15:33.045800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568fe866-25b0-476b-a5b2-3d83f99c15ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:15:33.046470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fe866-25b0-476b-a5b2-3d83f99c15ed None admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:15:33.046754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fe866-25b0-476b-a5b2-3d83f99c15ed None admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:15:33.047033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fe866-25b0-476b-a5b2-3d83f99c15ed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:15:33.047760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fe866-25b0-476b-a5b2-3d83f99c15ed None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:15:33.049161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568fe866-25b0-476b-a5b2-3d83f99c15ed None admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:15:33.066887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 643/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Fri Jul 15 19:15:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:15:42.907364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=30 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:15:43.063122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:35.956781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664d343-05a6-4179-983c-2c0f7140bb59 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:35.957373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664d343-05a6-4179-983c-2c0f7140bb59 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:35.957373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664d343-05a6-4179-983c-2c0f7140bb59 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:35.964518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca574e9-8c86-4bfc-8595-be09d3b8394b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:35.965018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca574e9-8c86-4bfc-8595-be09d3b8394b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:35.965113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca574e9-8c86-4bfc-8595-be09d3b8394b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:35.967628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 664/1307] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:36.049439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3ca574e9-8c86-4bfc-8595-be09d3b8394b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:36.050552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3ca574e9-8c86-4bfc-8595-be09d3b8394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:36.209219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 644/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:16:36.259338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2fd39-9738-4d08-83ed-28366a382790 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:36.259626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2fd39-9738-4d08-83ed-28366a382790 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:36.259899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2fd39-9738-4d08-83ed-28366a382790 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:36.261674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 665/1309] 10.222.0.17 () {54 vars in 964 bytes} [Fri Jul 15 19:16:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:36.267978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c39b-c99f-4484-b8f3-3cb1225cfbc2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:36.268405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c39b-c99f-4484-b8f3-3cb1225cfbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:36.268775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c39b-c99f-4484-b8f3-3cb1225cfbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:36.358753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1112c39b-c99f-4484-b8f3-3cb1225cfbc2 None None] MFA Rules not processed for user `9f135ce75fbc4b12977cc8f009083ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:36.360072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1112c39b-c99f-4484-b8f3-3cb1225cfbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:36.382285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:36.444106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 645/1310] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Jul 15 19:16:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:36.453693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2fd39-9738-4d08-83ed-28366a382790 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:36.455573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3ba2fd39-9738-4d08-83ed-28366a382790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:36.570670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2fd39-9738-4d08-83ed-28366a382790 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:36.650512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:36.651277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:36.651628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:36.651974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:36.653725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:36.712793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:36.776388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:36.778023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c892bdeb-611f-41d1-8006-6d52ccf1bfe0 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:36.798516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 666/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:39.128893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544581a7-112e-432e-b33c-146395701377 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:39.129425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544581a7-112e-432e-b33c-146395701377 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:39.129425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544581a7-112e-432e-b33c-146395701377 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:39.130843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 646/1312] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:39.136415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b4cfd-3511-4662-b672-c120e7979e86 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:39.136814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b4cfd-3511-4662-b672-c120e7979e86 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:39.136974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b4cfd-3511-4662-b672-c120e7979e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:39.212779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ca1b4cfd-3511-4662-b672-c120e7979e86 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:39.213984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ca1b4cfd-3511-4662-b672-c120e7979e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:39.322416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 667/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:39.362352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544581a7-112e-432e-b33c-146395701377 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:39.364106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-544581a7-112e-432e-b33c-146395701377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:39.487752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544581a7-112e-432e-b33c-146395701377 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:39.571883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:39.575671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:39.576592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:39.577225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:39.577910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:39.580364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:39.713587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:39.715073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bbbb0c-4a51-4cb8-b400-1f14279e1dfb service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:39.735709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 647/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:42.060440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07b0c7-97e1-435f-b385-593e162111cf None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:42.060976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07b0c7-97e1-435f-b385-593e162111cf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:42.060976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07b0c7-97e1-435f-b385-593e162111cf None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:42.062287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 668/1315] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:42.067289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a694757-a2c7-4707-9915-4b756c4f3969 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:42.067705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a694757-a2c7-4707-9915-4b756c4f3969 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:42.067848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a694757-a2c7-4707-9915-4b756c4f3969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:42.137830 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2a694757-a2c7-4707-9915-4b756c4f3969 None None] MFA Rules not processed for user `30cadd9bcfd84eaba7b31000da7993e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:42.139059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2a694757-a2c7-4707-9915-4b756c4f3969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:42.189486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:42.233442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 648/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:42.288145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d55cc-ec03-4c14-b85c-7825214d4d80 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:42.288541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d55cc-ec03-4c14-b85c-7825214d4d80 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:42.288933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4d55cc-ec03-4c14-b85c-7825214d4d80 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:42.290311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 669/1317] 10.222.0.17 () {54 vars in 964 bytes} [Fri Jul 15 19:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:42.296354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174176-e62d-4b09-9171-a0f7df790f75 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:42.296646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174176-e62d-4b09-9171-a0f7df790f75 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:42.296923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11174176-e62d-4b09-9171-a0f7df790f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:42.370677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-11174176-e62d-4b09-9171-a0f7df790f75 None None] MFA Rules not processed for user `9f135ce75fbc4b12977cc8f009083ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:42.371882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-11174176-e62d-4b09-9171-a0f7df790f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:42.447728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 649/1318] 10.222.0.17 () {58 vars in 1061 bytes} [Fri Jul 15 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:42.456870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d55cc-ec03-4c14-b85c-7825214d4d80 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:42.458388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dd4d55cc-ec03-4c14-b85c-7825214d4d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:42.468054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:42.560123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4d55cc-ec03-4c14-b85c-7825214d4d80 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:42.631293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:42.631864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:42.632182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:42.632461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:42.634258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:42.695407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:42.744354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:42.746011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eddff4c-bd68-44c6-8f62-c341d60badb1 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:42.764246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 670/1319] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:45.726080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5f332-5496-43a1-a0d6-b9e6f915552e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:45.726491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5f332-5496-43a1-a0d6-b9e6f915552e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:45.726637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5f332-5496-43a1-a0d6-b9e6f915552e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:45.728029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 650/1320] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:45.733094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9bb2-e13f-4aa2-9a93-a9580ad9531b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:45.733485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9bb2-e13f-4aa2-9a93-a9580ad9531b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:45.733647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca9bb2-e13f-4aa2-9a93-a9580ad9531b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:45.797705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-afca9bb2-e13f-4aa2-9a93-a9580ad9531b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:45.798895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-afca9bb2-e13f-4aa2-9a93-a9580ad9531b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:45.896809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 671/1321] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:45.940854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5f332-5496-43a1-a0d6-b9e6f915552e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:45.942409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89a5f332-5496-43a1-a0d6-b9e6f915552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:45.994251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:46.060584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5f332-5496-43a1-a0d6-b9e6f915552e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:46.133705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:46.134348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:46.134633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:46.134911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:46.136588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:46.230271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:46.244244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:46.245736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bbc7e-0757-4bdf-8287-2cfe5220f0a3 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:46.277829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 651/1322] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:49.041331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c8d45-cd71-4683-b7fd-5eff503ac66a None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:49.041854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c8d45-cd71-4683-b7fd-5eff503ac66a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:49.041854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7c8d45-cd71-4683-b7fd-5eff503ac66a None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:49.043218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 672/1323] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:49.048524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739fe6d-8bbb-4105-a353-7df693b208c2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:49.048963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739fe6d-8bbb-4105-a353-7df693b208c2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:49.049072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d739fe6d-8bbb-4105-a353-7df693b208c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:49.114432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d739fe6d-8bbb-4105-a353-7df693b208c2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:49.115617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d739fe6d-8bbb-4105-a353-7df693b208c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:49.212780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 652/1324] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:49.251464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c8d45-cd71-4683-b7fd-5eff503ac66a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:49.253082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4d7c8d45-cd71-4683-b7fd-5eff503ac66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:49.273580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:49.373365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7c8d45-cd71-4683-b7fd-5eff503ac66a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:49.457142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:49.457800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:49.458082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:49.458356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:49.460640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:49.539129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:49.580883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:49.582356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbc3bc2-2b26-4c5d-98b9-4db6a6bb00c4 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:49.609737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 673/1325] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:51.719860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bd009-d075-43a8-ac45-07d9d9e4e6a8 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:51.720332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bd009-d075-43a8-ac45-07d9d9e4e6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:51.720469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2bd009-d075-43a8-ac45-07d9d9e4e6a8 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:51.722005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 653/1326] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:51.727359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7e9d-9055-492a-8699-fed5f56cbd91 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:51.727784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7e9d-9055-492a-8699-fed5f56cbd91 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:51.727930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb7e9d-9055-492a-8699-fed5f56cbd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:51.793726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3deb7e9d-9055-492a-8699-fed5f56cbd91 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:51.794958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3deb7e9d-9055-492a-8699-fed5f56cbd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:51.890874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 674/1327] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:51.926960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bd009-d075-43a8-ac45-07d9d9e4e6a8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:51.929341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0f2bd009-d075-43a8-ac45-07d9d9e4e6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:51.993921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:52.038635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2bd009-d075-43a8-ac45-07d9d9e4e6a8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:52.114388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:52.115042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:52.115327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:52.115607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:52.118175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:52.226710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:52.229179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:52.231349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ec4d1-d444-41ea-be39-b94b8b697dc1 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:52.251432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 654/1328] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:54.319290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1e8b8-182a-422e-9600-b0cda2dde76e None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:54.319943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1e8b8-182a-422e-9600-b0cda2dde76e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:54.319943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1e8b8-182a-422e-9600-b0cda2dde76e None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:54.321260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 675/1329] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:54.326520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0d1d9-318b-44d9-9623-1750913fe12e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:54.326919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0d1d9-318b-44d9-9623-1750913fe12e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:54.327078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0d1d9-318b-44d9-9623-1750913fe12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:54.391843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-14e0d1d9-318b-44d9-9623-1750913fe12e None None] MFA Rules not processed for user `30cadd9bcfd84eaba7b31000da7993e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:54.393083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-14e0d1d9-318b-44d9-9623-1750913fe12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:54.494502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 655/1330] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:54.533110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1e8b8-182a-422e-9600-b0cda2dde76e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:54.534645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-41e1e8b8-182a-422e-9600-b0cda2dde76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:54.566173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:54.643119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1e8b8-182a-422e-9600-b0cda2dde76e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:54.723588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:54.724293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:54.724581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:54.724855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:54.727439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:54.810936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:54.842716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:54.844194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8043f7-9a3e-497a-aad6-613283053d64 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:54.862556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 676/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:57.118028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052429d-2958-4f7a-8aeb-eb1518190a26 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.118623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052429d-2958-4f7a-8aeb-eb1518190a26 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.118623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052429d-2958-4f7a-8aeb-eb1518190a26 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.120004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 656/1332] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:57.125223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986ccb6-3200-4996-ad23-45f4ed494dcd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.125676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986ccb6-3200-4996-ad23-45f4ed494dcd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.125774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986ccb6-3200-4996-ad23-45f4ed494dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.190642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0986ccb6-3200-4996-ad23-45f4ed494dcd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:57.191769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0986ccb6-3200-4996-ad23-45f4ed494dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:57.286772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 677/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:57.322794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052429d-2958-4f7a-8aeb-eb1518190a26 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:16:57.324386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6052429d-2958-4f7a-8aeb-eb1518190a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:57.395505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:57.430709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052429d-2958-4f7a-8aeb-eb1518190a26 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:16:57.504833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:16:57.505386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:57.505666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:57.505938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:57.507643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:57.615358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:16:57.617114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4876ef-d660-493c-bbcf-cf841e9cf117 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:16:57.626980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:16:57.635329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 657/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:16:59.782560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1246-473b-47e7-91ed-ea03db695f2c None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:59.783140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1246-473b-47e7-91ed-ea03db695f2c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:59.783407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc1246-473b-47e7-91ed-ea03db695f2c None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:59.785790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 678/1335] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:16:59.792221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe632bb-0f69-486b-9563-7e2439aff860 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:16:59.792659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe632bb-0f69-486b-9563-7e2439aff860 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:16:59.792789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe632bb-0f69-486b-9563-7e2439aff860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:16:59.859563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0fe632bb-0f69-486b-9563-7e2439aff860 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:16:59.860811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0fe632bb-0f69-486b-9563-7e2439aff860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:16:59.958947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 658/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:16:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:16:59.999315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc1246-473b-47e7-91ed-ea03db695f2c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:00.000884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-75fc1246-473b-47e7-91ed-ea03db695f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:00.044998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:00.109239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc1246-473b-47e7-91ed-ea03db695f2c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:00.180005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:00.180682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:00.180963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:00.181235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:00.182889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:00.264447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:00.290035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:00.291465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665627a7-11a0-4758-aa10-0042bd508f32 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:00.309085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 679/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:16:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:02.653441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292bc1-c506-4239-a6a5-bf63fe30415b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:02.653987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292bc1-c506-4239-a6a5-bf63fe30415b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:02.653987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2292bc1-c506-4239-a6a5-bf63fe30415b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:02.655441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 659/1338] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:02.660696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64384d-e811-46cf-b985-2e3d84cd304c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:02.661117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64384d-e811-46cf-b985-2e3d84cd304c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:02.661252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64384d-e811-46cf-b985-2e3d84cd304c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:02.724461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1c64384d-e811-46cf-b985-2e3d84cd304c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:02.726173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1c64384d-e811-46cf-b985-2e3d84cd304c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:02.821571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 680/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:02.858277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292bc1-c506-4239-a6a5-bf63fe30415b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:02.859813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d2292bc1-c506-4239-a6a5-bf63fe30415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:02.937577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:02.970523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2292bc1-c506-4239-a6a5-bf63fe30415b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:03.045400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:03.046053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:03.046340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:03.046620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:03.048311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:03.159299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:03.160775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3f75de-340e-4b87-abe5-2137d3cddbc5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:03.180548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 660/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:07.328391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e849c-fe26-4f46-9c08-7e5a860c13ac None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:07.328902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e849c-fe26-4f46-9c08-7e5a860c13ac None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:07.329015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e849c-fe26-4f46-9c08-7e5a860c13ac None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:07.330452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 681/1341] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:07.336179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5795428-4bc7-40e3-a9ea-59ed89b8b664 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:07.336600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5795428-4bc7-40e3-a9ea-59ed89b8b664 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:07.336766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5795428-4bc7-40e3-a9ea-59ed89b8b664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:07.340957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:07.404341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a5795428-4bc7-40e3-a9ea-59ed89b8b664 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:07.405543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a5795428-4bc7-40e3-a9ea-59ed89b8b664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:07.504787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 661/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:07.545361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e849c-fe26-4f46-9c08-7e5a860c13ac None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:07.546901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b67e849c-fe26-4f46-9c08-7e5a860c13ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:07.597321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:07.652680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e849c-fe26-4f46-9c08-7e5a860c13ac None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:07.733404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:07.734149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:07.734431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:07.734720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:07.736404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:07.837060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:07.853761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:07.855219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e4d33-2ed2-4125-a2d8-2be102b6032f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:07.873270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 682/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:10.500821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7c4ec-791c-4ad2-8db2-ad332423ed12 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:10.501377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7c4ec-791c-4ad2-8db2-ad332423ed12 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:10.501377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7c4ec-791c-4ad2-8db2-ad332423ed12 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:10.502801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 662/1344] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:10.508982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c6f36-fb22-4813-9ec8-376e13f3ed2c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:10.509533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c6f36-fb22-4813-9ec8-376e13f3ed2c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:10.509676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c6f36-fb22-4813-9ec8-376e13f3ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:10.584033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-477c6f36-fb22-4813-9ec8-376e13f3ed2c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:10.586030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-477c6f36-fb22-4813-9ec8-376e13f3ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:10.692254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 683/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:10.731800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7c4ec-791c-4ad2-8db2-ad332423ed12 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:10.733464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-07c7c4ec-791c-4ad2-8db2-ad332423ed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:10.822566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:10.840491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7c4ec-791c-4ad2-8db2-ad332423ed12 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:10.913584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:10.914163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:10.914446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:10.914725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:10.916391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:11.025840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:11.027217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96ff380-6675-47d7-bc7c-f78da08dca5c service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:11.045849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 663/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:15.292752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5aacb-277c-4ca1-a3a0-64e66ccbd4bc None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:15.293300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5aacb-277c-4ca1-a3a0-64e66ccbd4bc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:15.293300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc5aacb-277c-4ca1-a3a0-64e66ccbd4bc None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:15.294661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 684/1347] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:15.299891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202b2f-b340-4fbd-8988-d811c5ff74bb None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:15.300336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202b2f-b340-4fbd-8988-d811c5ff74bb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:15.300482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9202b2f-b340-4fbd-8988-d811c5ff74bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:15.326135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:15.387893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b9202b2f-b340-4fbd-8988-d811c5ff74bb None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:15.389152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b9202b2f-b340-4fbd-8988-d811c5ff74bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:15.495024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 664/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:15.535480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5aacb-277c-4ca1-a3a0-64e66ccbd4bc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:15.537578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6cc5aacb-277c-4ca1-a3a0-64e66ccbd4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:15.599872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:15.650406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc5aacb-277c-4ca1-a3a0-64e66ccbd4bc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:15.728244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:15.728899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:15.729178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:15.729458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:15.731162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:15.857850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:15.862844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:15.864345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc1963-d3e0-49ca-83ea-98c65968999b service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:15.884575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 685/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:20.784979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e4dae-e0d3-434d-bda5-31fb435bfaf0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:20.785505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e4dae-e0d3-434d-bda5-31fb435bfaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:20.785505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e4dae-e0d3-434d-bda5-31fb435bfaf0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:20.788378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 665/1350] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:20.793670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9b5b9-11dc-41e6-968d-bbf45315549b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:20.794431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9b5b9-11dc-41e6-968d-bbf45315549b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:20.794876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9b5b9-11dc-41e6-968d-bbf45315549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:20.915527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c2a9b5b9-11dc-41e6-968d-bbf45315549b None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:20.917109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c2a9b5b9-11dc-41e6-968d-bbf45315549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:21.033753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 686/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:17:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:17:25.066726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefcf8-7e6d-440a-bfc8-c4aa06135db1 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:25.067256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefcf8-7e6d-440a-bfc8-c4aa06135db1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:25.067518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaefcf8-7e6d-440a-bfc8-c4aa06135db1 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:25.069393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 666/1352] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:25.074850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c5853-55ea-46b1-b9e9-b6be4b9c5516 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:25.075295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c5853-55ea-46b1-b9e9-b6be4b9c5516 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:25.075413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c5853-55ea-46b1-b9e9-b6be4b9c5516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:25.129881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:25.153880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c28c5853-55ea-46b1-b9e9-b6be4b9c5516 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:25.155086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c28c5853-55ea-46b1-b9e9-b6be4b9c5516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:25.269235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 687/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:17:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:25.493145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=59 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:26.732154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82537d6d-1359-44f8-859c-cf706773343d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:26.732838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82537d6d-1359-44f8-859c-cf706773343d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:26.732838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82537d6d-1359-44f8-859c-cf706773343d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:26.734089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 667/1354] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:26.739333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6971-0d88-476d-801e-ded19c440ba9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:26.739799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6971-0d88-476d-801e-ded19c440ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:26.739893 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e6971-0d88-476d-801e-ded19c440ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:26.815696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c49e6971-0d88-476d-801e-ded19c440ba9 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:26.816913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c49e6971-0d88-476d-801e-ded19c440ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:26.859480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:26.924764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 688/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:17:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:17:30.321274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e23ca1-a039-42b7-872c-452c7965076f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:30.322244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e23ca1-a039-42b7-872c-452c7965076f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:30.322735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e23ca1-a039-42b7-872c-452c7965076f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:30.324447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 668/1356] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:30.330274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46259a4-20cf-447b-af0d-496dc1144a5e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:30.330742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46259a4-20cf-447b-af0d-496dc1144a5e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:30.331027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46259a4-20cf-447b-af0d-496dc1144a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:30.408378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b46259a4-20cf-447b-af0d-496dc1144a5e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:30.409840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b46259a4-20cf-447b-af0d-496dc1144a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:30.523887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 689/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:30.585388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557911be-8117-4d29-81ae-0da5108034ad None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:30.586166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557911be-8117-4d29-81ae-0da5108034ad None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:30.586621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557911be-8117-4d29-81ae-0da5108034ad None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:30.588531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 669/1358] 10.222.0.17 () {54 vars in 961 bytes} [Fri Jul 15 19:17:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:30.597764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34165-4410-43d4-a8c1-734c35c99954 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:30.598285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34165-4410-43d4-a8c1-734c35c99954 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:30.598822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b34165-4410-43d4-a8c1-734c35c99954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:30.626072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:30.701962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a1b34165-4410-43d4-a8c1-734c35c99954 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:30.703887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a1b34165-4410-43d4-a8c1-734c35c99954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:30.868174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 690/1359] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Jul 15 19:17:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:17:30.877982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557911be-8117-4d29-81ae-0da5108034ad None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:30.879500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-557911be-8117-4d29-81ae-0da5108034ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:31.060722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557911be-8117-4d29-81ae-0da5108034ad None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:31.151855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:31.197835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:31.198493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:31.198775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:31.199052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:31.200736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:31.302614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:31.304935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a35c7-68b8-45a6-a761-1cbfd7563764 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:31.324670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 670/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:33.552900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a0427-ee90-4b30-ae6d-63971c39043a None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:33.553408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a0427-ee90-4b30-ae6d-63971c39043a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:33.553408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a0427-ee90-4b30-ae6d-63971c39043a None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:33.556485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 691/1361] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:33.559998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00936f-51b1-4cb8-8304-4b53c2b90742 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:33.560435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00936f-51b1-4cb8-8304-4b53c2b90742 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:33.560588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00936f-51b1-4cb8-8304-4b53c2b90742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:33.628443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8b00936f-51b1-4cb8-8304-4b53c2b90742 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:33.629647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8b00936f-51b1-4cb8-8304-4b53c2b90742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:33.668652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:33.748324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 671/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:33.789683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a0427-ee90-4b30-ae6d-63971c39043a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:33.792290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b69a0427-ee90-4b30-ae6d-63971c39043a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:33.869656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:33.970312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a0427-ee90-4b30-ae6d-63971c39043a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:34.092360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:34.093015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:34.093301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:34.093580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:34.095243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:34.120129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:34.190639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:34.192298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442807ef-5ac2-453a-ac8d-cd284b7095f4 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:34.211957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 692/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:36.604387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f513df-4b91-4566-9978-86a4d7c9fe00 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:36.605249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f513df-4b91-4566-9978-86a4d7c9fe00 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:36.605529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f513df-4b91-4566-9978-86a4d7c9fe00 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:36.607075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 672/1364] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:36.612470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd449a9f-95df-46f4-b9db-7550e26ab5b2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:36.613245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd449a9f-95df-46f4-b9db-7550e26ab5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:36.613531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd449a9f-95df-46f4-b9db-7550e26ab5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:36.685034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bd449a9f-95df-46f4-b9db-7550e26ab5b2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:36.686273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bd449a9f-95df-46f4-b9db-7550e26ab5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:36.791886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:36.795132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 693/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:36.841186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccc02b-003f-4b39-98fa-2d56afb47385 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:36.841473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccc02b-003f-4b39-98fa-2d56afb47385 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:36.841746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebccc02b-003f-4b39-98fa-2d56afb47385 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:36.843452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 673/1366] 10.222.0.17 () {54 vars in 961 bytes} [Fri Jul 15 19:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:36.849116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b487-41ba-4445-8377-c084b4c9d807 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:36.849515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b487-41ba-4445-8377-c084b4c9d807 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:36.850694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b487-41ba-4445-8377-c084b4c9d807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:36.931852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8e72b487-41ba-4445-8377-c084b4c9d807 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:36.933169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8e72b487-41ba-4445-8377-c084b4c9d807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:37.058756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 694/1367] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Jul 15 19:17:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:37.066846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccc02b-003f-4b39-98fa-2d56afb47385 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:37.069148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ebccc02b-003f-4b39-98fa-2d56afb47385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:37.240120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebccc02b-003f-4b39-98fa-2d56afb47385 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:37.263325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:37.361837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:37.362669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:37.363057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:37.363433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:37.365227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:37.462404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:37.464215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7a81dd-520d-4bb3-ad87-b59a13ffc136 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:37.483358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 674/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:38.491324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c04c6-208a-43d6-8f98-aae6e1153d48 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:38.491906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c04c6-208a-43d6-8f98-aae6e1153d48 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:38.491906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c04c6-208a-43d6-8f98-aae6e1153d48 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:38.493762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 695/1369] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:38.498870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170b737-9680-4f95-b43c-47bb80f90528 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:38.499264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170b737-9680-4f95-b43c-47bb80f90528 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:38.499436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170b737-9680-4f95-b43c-47bb80f90528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:38.531169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:38.569427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3170b737-9680-4f95-b43c-47bb80f90528 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:38.570660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3170b737-9680-4f95-b43c-47bb80f90528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:38.675153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 675/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:39.020720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c70036-c031-4fe5-98a2-fe9cde3426e0 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:39.021011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c70036-c031-4fe5-98a2-fe9cde3426e0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:39.021289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c70036-c031-4fe5-98a2-fe9cde3426e0 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:39.023857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 696/1371] 10.222.0.17 () {54 vars in 963 bytes} [Fri Jul 15 19:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:39.029004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc450d-c83c-4183-b3f1-61d939282207 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:39.029304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc450d-c83c-4183-b3f1-61d939282207 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:39.029561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc450d-c83c-4183-b3f1-61d939282207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:39.116900 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-01cc450d-c83c-4183-b3f1-61d939282207 None None] MFA Rules not processed for user `43fda9627eb84093bb59ceb1a553baf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:39.118732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-01cc450d-c83c-4183-b3f1-61d939282207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:39.169971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:39.205459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 676/1372] 10.222.0.17 () {58 vars in 1060 bytes} [Fri Jul 15 19:17:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:39.213436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c70036-c031-4fe5-98a2-fe9cde3426e0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:39.215155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e4c70036-c031-4fe5-98a2-fe9cde3426e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:39.275020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:39.334772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c70036-c031-4fe5-98a2-fe9cde3426e0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:39.414900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:39.415723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:39.416231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:39.416616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:39.418375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:39.531238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:39.546825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:39.548304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42a6c2-236a-4929-b2b1-996e11bf8c23 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:39.567825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 697/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:17:39.611221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd3705-0b30-41f6-8fe0-feb145356933 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:39.611684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd3705-0b30-41f6-8fe0-feb145356933 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:39.611684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bd3705-0b30-41f6-8fe0-feb145356933 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:39.613102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 677/1374] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:39.621663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7246ec-d984-4d16-9381-cc6bcaadfd23 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:39.622021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7246ec-d984-4d16-9381-cc6bcaadfd23 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:39.622170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7246ec-d984-4d16-9381-cc6bcaadfd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:39.688503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0e7246ec-d984-4d16-9381-cc6bcaadfd23 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:39.689821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0e7246ec-d984-4d16-9381-cc6bcaadfd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:39.812518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 698/1375] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:39.848589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd3705-0b30-41f6-8fe0-feb145356933 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:39.850358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d7bd3705-0b30-41f6-8fe0-feb145356933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:40.000325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bd3705-0b30-41f6-8fe0-feb145356933 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:40.049677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:40.112007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:40.112782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:40.113136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:40.113499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:40.115249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:40.204736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:40.206456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d166e-6864-4ba8-a08d-137539d98589 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:40.227769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 678/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:42.297839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb090-4746-4611-a5b8-f01fe69a31e1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:42.298339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb090-4746-4611-a5b8-f01fe69a31e1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:42.298339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69abb090-4746-4611-a5b8-f01fe69a31e1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:42.299701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 699/1377] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:42.306284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75faee4-c6f4-4415-b222-a7c3e3e4e666 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:42.306722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75faee4-c6f4-4415-b222-a7c3e3e4e666 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:42.306979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75faee4-c6f4-4415-b222-a7c3e3e4e666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:42.379597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f75faee4-c6f4-4415-b222-a7c3e3e4e666 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:42.380957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f75faee4-c6f4-4415-b222-a7c3e3e4e666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:42.389039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:42.488411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 679/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:42.534814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb090-4746-4611-a5b8-f01fe69a31e1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:42.536412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-69abb090-4746-4611-a5b8-f01fe69a31e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:42.620452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:42.708313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69abb090-4746-4611-a5b8-f01fe69a31e1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:42.830426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:42.831082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:42.831360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:42.831641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:42.833380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:42.854103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:42.948188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:42.949828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915dab72-ad09-43dd-a180-1f9cee9d5e02 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:42.968686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 700/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:45.058316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa770609-9de1-4135-b046-636e0755bacb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:45.058868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa770609-9de1-4135-b046-636e0755bacb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:45.058868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa770609-9de1-4135-b046-636e0755bacb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:45.060316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 680/1380] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:45.065430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ced3b9-de83-4b2f-a32b-1c9d5999e7cd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:45.065891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ced3b9-de83-4b2f-a32b-1c9d5999e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:45.065996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ced3b9-de83-4b2f-a32b-1c9d5999e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:45.138093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e6ced3b9-de83-4b2f-a32b-1c9d5999e7cd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:45.139737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e6ced3b9-de83-4b2f-a32b-1c9d5999e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:45.250951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:45.261989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 701/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:45.301349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa770609-9de1-4135-b046-636e0755bacb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:45.303117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-aa770609-9de1-4135-b046-636e0755bacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:45.461169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:45.476144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa770609-9de1-4135-b046-636e0755bacb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:45.601802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:45.602454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:45.602733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:45.603007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:45.604664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:45.698758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:45.700454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352401da-85bd-4f4a-acf1-5a75d7484a39 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:45.711912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:45.721733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 681/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:48.043098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3337e-9a70-4137-8fc4-0b098c6405d9 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:48.043610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3337e-9a70-4137-8fc4-0b098c6405d9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:48.043610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3337e-9a70-4137-8fc4-0b098c6405d9 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:48.045079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 702/1383] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:48.050534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f333e-36d0-4bfe-bdc5-8b65892ce0f8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:48.051133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f333e-36d0-4bfe-bdc5-8b65892ce0f8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:48.051133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8f333e-36d0-4bfe-bdc5-8b65892ce0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:48.122591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0b8f333e-36d0-4bfe-bdc5-8b65892ce0f8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:48.123862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0b8f333e-36d0-4bfe-bdc5-8b65892ce0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:48.224682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 682/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:48.269168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3337e-9a70-4137-8fc4-0b098c6405d9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:48.271235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bec3337e-9a70-4137-8fc4-0b098c6405d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:48.437845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3337e-9a70-4137-8fc4-0b098c6405d9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:48.512679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:48.565779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:48.566492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:48.566791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:48.567083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:48.569109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:48.665062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:48.666703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ea44a-94d3-4991-a6a2-95e5b0f3d76e service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:48.685170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 703/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:50.978646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff531-ddca-40d1-ade9-812d36ff2c10 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:50.979317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff531-ddca-40d1-ade9-812d36ff2c10 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:50.979317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696ff531-ddca-40d1-ade9-812d36ff2c10 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:50.980626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 683/1386] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:50.985960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29bfac-b2cb-43bc-80a5-ad1edcf092cb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:50.986422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29bfac-b2cb-43bc-80a5-ad1edcf092cb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:50.986762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a29bfac-b2cb-43bc-80a5-ad1edcf092cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:51.065335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5a29bfac-b2cb-43bc-80a5-ad1edcf092cb None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:51.066954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5a29bfac-b2cb-43bc-80a5-ad1edcf092cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:51.104749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:51.175477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 704/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:51.220326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ff531-ddca-40d1-ade9-812d36ff2c10 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:51.221942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-696ff531-ddca-40d1-ade9-812d36ff2c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:51.315404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:51.406017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696ff531-ddca-40d1-ade9-812d36ff2c10 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:51.527647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:51.528369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:51.528667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:51.528964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:51.530698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:51.594730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:51.659182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:51.660865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c28719e-b9f3-44aa-a01b-99142bbce760 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:51.679820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 684/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:53.934838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52cae-946b-4dd7-b826-1e695270f8ee None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:53.935407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52cae-946b-4dd7-b826-1e695270f8ee None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:53.935407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea52cae-946b-4dd7-b826-1e695270f8ee None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:53.936793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 705/1389] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:53.941937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12551b-a2a2-4da0-bc78-03b59b40de49 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:53.942366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12551b-a2a2-4da0-bc78-03b59b40de49 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:53.942495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12551b-a2a2-4da0-bc78-03b59b40de49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:54.007615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2f12551b-a2a2-4da0-bc78-03b59b40de49 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:54.008829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2f12551b-a2a2-4da0-bc78-03b59b40de49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:54.105251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 685/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:54.149154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea52cae-946b-4dd7-b826-1e695270f8ee None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:54.150692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dea52cae-946b-4dd7-b826-1e695270f8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:54.297679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea52cae-946b-4dd7-b826-1e695270f8ee None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:54.314511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:54.411599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:54.412174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:54.412458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:54.413284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:54.415217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:54.506569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:54.508449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcce41a-6a55-4075-9f49-eee9a17de7ce service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:54.527232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 706/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:54.722066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ff8f6-c121-42d3-8507-d06d1511f142 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:54.722295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ff8f6-c121-42d3-8507-d06d1511f142 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:54.722576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ff8f6-c121-42d3-8507-d06d1511f142 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:54.723995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 686/1392] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:54.729963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365074-dea4-42ee-a82b-23c7e2e4e0a3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:54.730263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365074-dea4-42ee-a82b-23c7e2e4e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:54.730546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8365074-dea4-42ee-a82b-23c7e2e4e0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:54.757678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:54.802972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f8365074-dea4-42ee-a82b-23c7e2e4e0a3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:54.804779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f8365074-dea4-42ee-a82b-23c7e2e4e0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:54.902574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 707/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:54.958956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca2ca0-6129-44bb-8f28-c85df9fee654 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:54.959394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca2ca0-6129-44bb-8f28-c85df9fee654 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:54.959615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca2ca0-6129-44bb-8f28-c85df9fee654 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:54.961259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 687/1394] 10.222.0.17 () {54 vars in 960 bytes} [Fri Jul 15 19:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:54.966265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0708a2-bee5-44c0-bba7-388032b303f5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:54.966669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0708a2-bee5-44c0-bba7-388032b303f5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:54.966827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0708a2-bee5-44c0-bba7-388032b303f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:55.044184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1b0708a2-bee5-44c0-bba7-388032b303f5 None None] MFA Rules not processed for user `6802452aab2f43d4b965be689758b611`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:55.045335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1b0708a2-bee5-44c0-bba7-388032b303f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:55.087852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:55.125055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 708/1395] 10.222.0.17 () {58 vars in 1057 bytes} [Fri Jul 15 19:17:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:55.132932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca2ca0-6129-44bb-8f28-c85df9fee654 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:55.134492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-afca2ca0-6129-44bb-8f28-c85df9fee654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:55.149340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:55.245578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca2ca0-6129-44bb-8f28-c85df9fee654 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:55.323078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:55.323648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:55.323932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:55.324246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:55.325942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:55.394939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:55.436508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:55.438079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413f8fa-28ca-4cb2-92ca-65332221f3e0 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:55.457236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 688/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:17:56.176562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c97c-7bfa-4d5d-b659-716d4a84bf08 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:56.177043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c97c-7bfa-4d5d-b659-716d4a84bf08 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:56.177140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a8c97c-7bfa-4d5d-b659-716d4a84bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:56.243495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a8a8c97c-7bfa-4d5d-b659-716d4a84bf08 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:56.244882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a8a8c97c-7bfa-4d5d-b659-716d4a84bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:56.334721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 709/1397] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:17:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:56.380518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e28080-5cdb-4566-ac4b-657e9ecf60ca None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:56.380958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e28080-5cdb-4566-ac4b-657e9ecf60ca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:56.381218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e28080-5cdb-4566-ac4b-657e9ecf60ca None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:56.383722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 689/1398] 10.222.0.17 () {54 vars in 942 bytes} [Fri Jul 15 19:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:56.389975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:56.390271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:56.390576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:56.460437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:56.467211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] MFA Rules not processed for user `8b3c45809bac4828bcde30494ab7a0d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:56.468443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:56.582101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 710/1399] 10.222.0.17 () {58 vars in 1039 bytes} [Fri Jul 15 19:17:56 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:56.587164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078faf-bd34-4db0-8a86-7e1f20b1e27a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:56.587453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078faf-bd34-4db0-8a86-7e1f20b1e27a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:56.587734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67078faf-bd34-4db0-8a86-7e1f20b1e27a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:56.589137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 690/1400] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:56.593310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:56.594049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:56.594340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:56.594619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:56.602961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22b7cde-49a1-47d3-a6d5-5f0e12a254b5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:56.660604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:56.661812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:56.715139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:56.715865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:56.716199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:56.716481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:56.718891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:56.721905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:56.800160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 691/1401] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:17:56.827567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:56.829820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d11bf2-e1f9-405c-8860-609bd4fbfdc3 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:56.843285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 711/1402] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:17:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:17:56.846884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:56.847429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:56.877051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:56.987576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b5533-5896-48f0-b5c3-64e235ad02f2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:57.116658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:57.117312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:57.117591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:57.117863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:57.119908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:57.139709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:57.221945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:57.223636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296be6d-1b7f-43d0-a7b8-f1eb218d0a0d service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:57.245942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 692/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:17:59.299966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f2e3f-161e-48fe-87eb-6596a57c3056 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:59.300552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f2e3f-161e-48fe-87eb-6596a57c3056 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:59.300552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f2e3f-161e-48fe-87eb-6596a57c3056 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:59.301906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 712/1404] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:17:59.307854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43815d-7284-4cab-ac97-0d0eaa6c636f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:59.308337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43815d-7284-4cab-ac97-0d0eaa6c636f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:59.308446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43815d-7284-4cab-ac97-0d0eaa6c636f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:59.377896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-db43815d-7284-4cab-ac97-0d0eaa6c636f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:17:59.379395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-db43815d-7284-4cab-ac97-0d0eaa6c636f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:59.480797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:59.489418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 693/1405] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:17:59.533781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f2e3f-161e-48fe-87eb-6596a57c3056 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:17:59.535379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-143f2e3f-161e-48fe-87eb-6596a57c3056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:59.702761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:17:59.706580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f2e3f-161e-48fe-87eb-6596a57c3056 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:17:59.833771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:17:59.834427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:17:59.834740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:17:59.835075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:17:59.836769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:17:59.943765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:17:59.944576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb54a922-2e14-457b-bf78-f3bfab82edd2 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:17:59.964529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 713/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:02.047113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a06ad-9de0-4fec-8b45-326a2e7e62e0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:02.047624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a06ad-9de0-4fec-8b45-326a2e7e62e0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:02.047624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7a06ad-9de0-4fec-8b45-326a2e7e62e0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:02.052130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 694/1407] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:02.055022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1dfd-605d-4dff-8efe-3db5816305ae None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:02.055449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1dfd-605d-4dff-8efe-3db5816305ae None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:02.055796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e1dfd-605d-4dff-8efe-3db5816305ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:02.059109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:02.191353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-682e1dfd-605d-4dff-8efe-3db5816305ae None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:02.192652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-682e1dfd-605d-4dff-8efe-3db5816305ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:02.320134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 714/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:02.362546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a06ad-9de0-4fec-8b45-326a2e7e62e0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:02.364408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3b7a06ad-9de0-4fec-8b45-326a2e7e62e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:02.534173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7a06ad-9de0-4fec-8b45-326a2e7e62e0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:02.605146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:02.658723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:02.659912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:02.660409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:02.660889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:02.662606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:02.771103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:02.773729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ec257-b2d9-4b69-8186-00e2da1ea812 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:02.793791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 695/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:12.313313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=64 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:12.791035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=64 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:18.974143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6d8c-215e-4f18-911c-86c44693f033 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:18.974703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6d8c-215e-4f18-911c-86c44693f033 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:18.974703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6d8c-215e-4f18-911c-86c44693f033 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:18.976730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 715/1410] 10.222.0.17 () {54 vars in 918 bytes} [Fri Jul 15 19:18:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:18.982081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05407f97-4a86-43e9-baf0-94ebb66e93a6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:18.982475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05407f97-4a86-43e9-baf0-94ebb66e93a6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:18.982638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05407f97-4a86-43e9-baf0-94ebb66e93a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:19.056186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-05407f97-4a86-43e9-baf0-94ebb66e93a6 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:19.057312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-05407f97-4a86-43e9-baf0-94ebb66e93a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:19.160804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 696/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Jul 15 19:18:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:19.175474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3b20-c897-434e-841f-3f353616f4be None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:19.175761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3b20-c897-434e-841f-3f353616f4be None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:19.176040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f3b20-c897-434e-841f-3f353616f4be None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:19.177443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 716/1412] 10.222.0.17 () {54 vars in 918 bytes} [Fri Jul 15 19:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:19.182912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6420648-8ef4-424d-97f7-6d7325c6e9d2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:19.183214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6420648-8ef4-424d-97f7-6d7325c6e9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:19.183491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6420648-8ef4-424d-97f7-6d7325c6e9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:19.253756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:19.256805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f6420648-8ef4-424d-97f7-6d7325c6e9d2 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:19.258601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f6420648-8ef4-424d-97f7-6d7325c6e9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:19.375008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 697/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Fri Jul 15 19:18:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:18:21.831191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58568d1-4783-4dbd-b2b0-ac92ff812c07 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.831703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58568d1-4783-4dbd-b2b0-ac92ff812c07 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.831806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58568d1-4783-4dbd-b2b0-ac92ff812c07 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.833198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:21.839036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64181fa-bb73-4738-80f9-18d1c4ca0d93 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:21.839461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64181fa-bb73-4738-80f9-18d1c4ca0d93 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:21.839591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64181fa-bb73-4738-80f9-18d1c4ca0d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:21.902930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e64181fa-bb73-4738-80f9-18d1c4ca0d93 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:21.904159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e64181fa-bb73-4738-80f9-18d1c4ca0d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:21.957833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:22.001236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:22.045014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58568d1-4783-4dbd-b2b0-ac92ff812c07 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:22.046713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b58568d1-4783-4dbd-b2b0-ac92ff812c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:22.194899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58568d1-4783-4dbd-b2b0-ac92ff812c07 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:22.265729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:22.308367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:22.309018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:22.309297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:22.309571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:22.311214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:22.401552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:22.403184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d060555a-f326-4ac3-9c12-d1177c267c82 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:22.421122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 718/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:23.323094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff196e2-fc45-4372-bfbd-9b930566f2e9 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:23.323705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff196e2-fc45-4372-bfbd-9b930566f2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:23.323705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff196e2-fc45-4372-bfbd-9b930566f2e9 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:23.325047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:23.329870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed39a05-b31a-463a-9b09-e5a52b1448eb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:23.330334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed39a05-b31a-463a-9b09-e5a52b1448eb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:23.330439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed39a05-b31a-463a-9b09-e5a52b1448eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:23.405399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0ed39a05-b31a-463a-9b09-e5a52b1448eb None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:23.406607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0ed39a05-b31a-463a-9b09-e5a52b1448eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:23.433609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:23.504316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:23.653845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff196e2-fc45-4372-bfbd-9b930566f2e9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:23.655374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8ff196e2-fc45-4372-bfbd-9b930566f2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:23.800838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff196e2-fc45-4372-bfbd-9b930566f2e9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:23.840543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:23.914644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:23.915215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:23.915499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:23.915777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:23.917452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:24.004675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:24.006266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbc1d9-b69f-4224-929d-95c1a6c40cf2 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:24.024674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 700/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:24.910672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5be8fa-8a4d-4ec5-8581-69a896aae5b3 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:24.911223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5be8fa-8a4d-4ec5-8581-69a896aae5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:24.911223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5be8fa-8a4d-4ec5-8581-69a896aae5b3 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:24.912701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:24.917964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036d45-ab8e-451b-9526-d269e4efa7fc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:24.918392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036d45-ab8e-451b-9526-d269e4efa7fc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:24.918504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0036d45-ab8e-451b-9526-d269e4efa7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:24.974494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:24.998406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a0036d45-ab8e-451b-9526-d269e4efa7fc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:24.999619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a0036d45-ab8e-451b-9526-d269e4efa7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:25.096655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:25.252581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5be8fa-8a4d-4ec5-8581-69a896aae5b3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:25.254129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1c5be8fa-8a4d-4ec5-8581-69a896aae5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:25.398830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5be8fa-8a4d-4ec5-8581-69a896aae5b3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:25.411224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:25.512665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:25.513318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:25.513599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:25.513873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:25.515515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:25.602140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:25.603741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23917121-dee3-4c14-8b37-4dadd9567172 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:25.622463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 721/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:26.482805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ca25-643b-409a-a133-e180b651260d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.483394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ca25-643b-409a-a133-e180b651260d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.483394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d88ca25-643b-409a-a133-e180b651260d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.484788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:26.489741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916bca-5262-4101-b68b-22d5cbfb3e84 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:26.490139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916bca-5262-4101-b68b-22d5cbfb3e84 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:26.490304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39916bca-5262-4101-b68b-22d5cbfb3e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:26.509463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:26.567378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-39916bca-5262-4101-b68b-22d5cbfb3e84 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:26.568619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-39916bca-5262-4101-b68b-22d5cbfb3e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:26.666174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:26.824567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ca25-643b-409a-a133-e180b651260d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:26.826097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1d88ca25-643b-409a-a133-e180b651260d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:26.937078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:26.974350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d88ca25-643b-409a-a133-e180b651260d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:27.086860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:27.087435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:27.087722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:27.088000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:27.089695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:27.169744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:27.184094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:27.185788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2b357-cdf9-44e0-9881-e7f067f16429 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:27.204098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 703/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:28.067249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab02b11-650e-401f-8576-8bd362055452 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:28.067779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab02b11-650e-401f-8576-8bd362055452 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:28.067779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab02b11-650e-401f-8576-8bd362055452 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:28.069147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:28.074738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a42d-d70e-4831-b10e-eba37dd6e57d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:28.075161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a42d-d70e-4831-b10e-eba37dd6e57d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:28.075267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a42d-d70e-4831-b10e-eba37dd6e57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:28.152330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f206a42d-d70e-4831-b10e-eba37dd6e57d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:28.153578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f206a42d-d70e-4831-b10e-eba37dd6e57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:28.247718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:28.398087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab02b11-650e-401f-8576-8bd362055452 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:28.399747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cab02b11-650e-401f-8576-8bd362055452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:28.483488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:28.544929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab02b11-650e-401f-8576-8bd362055452 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:28.654662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:28.655266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:28.655549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:28.655825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:28.657529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:28.707596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:28.748811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:28.750429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe8fa3c-7602-4679-a8a1-a0e305bcc3d8 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:28.768624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 724/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:29.570630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d453307-a6ed-4f9d-a95c-905f72f72f33 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.571244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d453307-a6ed-4f9d-a95c-905f72f72f33 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.571244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d453307-a6ed-4f9d-a95c-905f72f72f33 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.572599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:29.577402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c810154-aab3-4387-a1db-c51bfb5803f4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:29.577855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c810154-aab3-4387-a1db-c51bfb5803f4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:29.577969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c810154-aab3-4387-a1db-c51bfb5803f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:29.654599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3c810154-aab3-4387-a1db-c51bfb5803f4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:29.655796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3c810154-aab3-4387-a1db-c51bfb5803f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:29.752332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:29.900146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d453307-a6ed-4f9d-a95c-905f72f72f33 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:29.901667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8d453307-a6ed-4f9d-a95c-905f72f72f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:29.950340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:30.046382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d453307-a6ed-4f9d-a95c-905f72f72f33 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:30.156819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:30.157386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:30.157667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:30.157941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:30.159569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:30.167659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:30.250323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:30.251929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a1750-0af5-4b27-ae4e-ea8635ff868a service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:30.270325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 706/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:31.089882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c617c-ec01-4e64-a39e-65d8977ef0f2 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:31.090444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c617c-ec01-4e64-a39e-65d8977ef0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:31.090444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c617c-ec01-4e64-a39e-65d8977ef0f2 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:31.091729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:31.100846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236528-8801-4cfa-a87d-5bbdecb2753a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:31.101738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236528-8801-4cfa-a87d-5bbdecb2753a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:31.101738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5236528-8801-4cfa-a87d-5bbdecb2753a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:31.184685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a5236528-8801-4cfa-a87d-5bbdecb2753a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:31.185910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a5236528-8801-4cfa-a87d-5bbdecb2753a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:31.271305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:31.304688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:31.476139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c617c-ec01-4e64-a39e-65d8977ef0f2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:31.477671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-061c617c-ec01-4e64-a39e-65d8977ef0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:31.501539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:31.652884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c617c-ec01-4e64-a39e-65d8977ef0f2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:31.762353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:31.793311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:31.793979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:31.794264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:31.794536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:31.796244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:31.903545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:31.905221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b8509-74d2-42d7-abb5-9bdbc887e315 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:31.925434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 727/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:32.738131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbb623-9613-4bd5-a127-8872bb342b64 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:32.738731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbb623-9613-4bd5-a127-8872bb342b64 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:32.738731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbb623-9613-4bd5-a127-8872bb342b64 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:32.740127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:32.745637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5d84-c645-4e5f-836a-3eac97466515 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:32.746103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5d84-c645-4e5f-836a-3eac97466515 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:32.746209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e5d84-c645-4e5f-836a-3eac97466515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:32.825313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5f2e5d84-c645-4e5f-836a-3eac97466515 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:32.826512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5f2e5d84-c645-4e5f-836a-3eac97466515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:32.867726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:32.923366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:33.075557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbb623-9613-4bd5-a127-8872bb342b64 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:33.077010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-52dbb623-9613-4bd5-a127-8872bb342b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:33.222533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbb623-9613-4bd5-a127-8872bb342b64 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:33.277008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:33.336162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:33.336759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:33.337042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:33.337320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:33.338971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:33.426840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:33.428467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50b8b6-7433-4e15-b42d-17b58a2a7d90 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:33.447297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 709/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:34.252135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af63bbc-34fd-4860-a747-d5032ebba8f6 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.252694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af63bbc-34fd-4860-a747-d5032ebba8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.252694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af63bbc-34fd-4860-a747-d5032ebba8f6 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.254055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:34.259062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9544e90-d11c-4a3f-93e1-a0f579d9a386 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.259487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9544e90-d11c-4a3f-93e1-a0f579d9a386 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.259623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9544e90-d11c-4a3f-93e1-a0f579d9a386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.334021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d9544e90-d11c-4a3f-93e1-a0f579d9a386 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:34.335736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d9544e90-d11c-4a3f-93e1-a0f579d9a386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:34.337765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:34.432494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:18:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:18:34.583894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af63bbc-34fd-4860-a747-d5032ebba8f6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:34.586113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8af63bbc-34fd-4860-a747-d5032ebba8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:34.732837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af63bbc-34fd-4860-a747-d5032ebba8f6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:34.760537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:34.845299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:34.845908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:34.846188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:34.846461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:34.848109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:34.936966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:34.938643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f5079-f7fe-44f9-8634-e132989ac7ed service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:34.956371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 730/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:36.887545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7673-d8a1-49f7-9db7-847c335d6d56 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:36.888063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7673-d8a1-49f7-9db7-847c335d6d56 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:36.888185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb7673-d8a1-49f7-9db7-847c335d6d56 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:36.889501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:36.894714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15f450-d706-4b8d-9a9b-dfcb58b6be8a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:36.895144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15f450-d706-4b8d-9a9b-dfcb58b6be8a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:36.895259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15f450-d706-4b8d-9a9b-dfcb58b6be8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:36.935099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:36.959641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6c15f450-d706-4b8d-9a9b-dfcb58b6be8a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:36.960884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c15f450-d706-4b8d-9a9b-dfcb58b6be8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:37.056060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:37.095389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb7673-d8a1-49f7-9db7-847c335d6d56 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:37.096966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0fcb7673-d8a1-49f7-9db7-847c335d6d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:37.203893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb7673-d8a1-49f7-9db7-847c335d6d56 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:37.235754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:37.277557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:37.278205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:37.278493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:37.278769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:37.280454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:37.388288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:37.389750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25953821-8db1-4694-8418-184cc879ecae service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:37.408483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 712/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:39.567133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ef6ca-1e61-44cd-a07b-22bd740d7ec2 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:39.567700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ef6ca-1e61-44cd-a07b-22bd740d7ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:39.567700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ef6ca-1e61-44cd-a07b-22bd740d7ec2 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:39.569221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:39.574805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e74018-659c-4a23-b9f4-7865f0e44bf0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:39.575228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e74018-659c-4a23-b9f4-7865f0e44bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:39.575361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e74018-659c-4a23-b9f4-7865f0e44bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:39.640780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-26e74018-659c-4a23-b9f4-7865f0e44bf0 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:39.641965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-26e74018-659c-4a23-b9f4-7865f0e44bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:39.654884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:39.738260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:39.777469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ef6ca-1e61-44cd-a07b-22bd740d7ec2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:39.778995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f24ef6ca-1e61-44cd-a07b-22bd740d7ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:39.882682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ef6ca-1e61-44cd-a07b-22bd740d7ec2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:39.894526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:39.954773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:39.955421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:39.955702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:39.955976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:39.957651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:40.065463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:40.066902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6d4ba-53b8-4889-a073-2f72dfcddd1a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:40.084438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 733/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:42.412602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d8d0e-0156-42d5-8a3b-e19110c10e5a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.413690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d8d0e-0156-42d5-8a3b-e19110c10e5a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.413690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d8d0e-0156-42d5-8a3b-e19110c10e5a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.415210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:42.420580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd075b-63fb-4e3b-82d4-6cd2f69d2437 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.421045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd075b-63fb-4e3b-82d4-6cd2f69d2437 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.421157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd075b-63fb-4e3b-82d4-6cd2f69d2437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.467137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:42.485153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-95dd075b-63fb-4e3b-82d4-6cd2f69d2437 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:42.486348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-95dd075b-63fb-4e3b-82d4-6cd2f69d2437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:42.582827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:42.622829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d8d0e-0156-42d5-8a3b-e19110c10e5a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:42.624463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1e8d8d0e-0156-42d5-8a3b-e19110c10e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:42.727917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d8d0e-0156-42d5-8a3b-e19110c10e5a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:42.770527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:42.805159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:42.805834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:42.806118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:42.806394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:42.808039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:42.916583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:42.918027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1e085-0bd5-4ac3-86a2-29d012b387df service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:42.936887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 715/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:45.048799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02a47-e1ff-4160-a749-57060ba8620a None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.049309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02a47-e1ff-4160-a749-57060ba8620a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.049309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02a47-e1ff-4160-a749-57060ba8620a None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.050649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:45.056472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88609bb3-1c11-4096-9f5b-4a87f30454c7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.056897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88609bb3-1c11-4096-9f5b-4a87f30454c7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.057034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88609bb3-1c11-4096-9f5b-4a87f30454c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.119858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-88609bb3-1c11-4096-9f5b-4a87f30454c7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:45.121351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-88609bb3-1c11-4096-9f5b-4a87f30454c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:45.141768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:45.218302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:45.257103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02a47-e1ff-4160-a749-57060ba8620a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:45.258726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1eb02a47-e1ff-4160-a749-57060ba8620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:45.363606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02a47-e1ff-4160-a749-57060ba8620a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:45.384743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:45.442146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:45.442799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:45.443085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:45.443365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:45.445292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:45.556154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:45.557612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f741514-8bf1-46bf-a902-565536bdc069 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:45.575568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 736/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:47.654428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b8898-3072-4b33-a9eb-5cafb5550e57 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:47.655029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b8898-3072-4b33-a9eb-5cafb5550e57 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:47.655029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b8898-3072-4b33-a9eb-5cafb5550e57 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:47.656452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:47.661651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f35b57-2973-4e11-8bba-7009dbba6066 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:47.662056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f35b57-2973-4e11-8bba-7009dbba6066 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:47.662216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f35b57-2973-4e11-8bba-7009dbba6066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:47.723817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:47.727176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a5f35b57-2973-4e11-8bba-7009dbba6066 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:47.728930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a5f35b57-2973-4e11-8bba-7009dbba6066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:47.826754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:47.865209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b8898-3072-4b33-a9eb-5cafb5550e57 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:47.866798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-046b8898-3072-4b33-a9eb-5cafb5550e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:47.968759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b8898-3072-4b33-a9eb-5cafb5550e57 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:48.016517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:48.041678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:48.042324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:48.042609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:48.042881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:48.044548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:48.154995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:48.156476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57381273-325c-4ed2-b6da-951eb635cf16 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:48.174178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 718/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:50.248570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31093723-86c6-410c-9e31-238f29af4cba None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.249112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31093723-86c6-410c-9e31-238f29af4cba None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.249206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31093723-86c6-410c-9e31-238f29af4cba None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.250539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:50.255890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab168-1cbb-474e-bf8d-cb44c7b40fc4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.256351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab168-1cbb-474e-bf8d-cb44c7b40fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.256457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cab168-1cbb-474e-bf8d-cb44c7b40fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.322045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d9cab168-1cbb-474e-bf8d-cb44c7b40fc4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:50.323137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d9cab168-1cbb-474e-bf8d-cb44c7b40fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:50.351287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:50.419392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:50.457720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31093723-86c6-410c-9e31-238f29af4cba None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:50.459292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-31093723-86c6-410c-9e31-238f29af4cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:50.562442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31093723-86c6-410c-9e31-238f29af4cba None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:50.591453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:50.644211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:50.644861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:50.645138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:50.645407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:50.647051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:50.755069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:50.756571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d7187-cc64-4acb-bb4d-5f4261c9eecb service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:50.776487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 739/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:52.805709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98754891-bc7b-41ca-bb4b-a906ddb669bb None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:52.806208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98754891-bc7b-41ca-bb4b-a906ddb669bb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:52.806326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98754891-bc7b-41ca-bb4b-a906ddb669bb None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:52.807645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:52.812950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7593-4db2-49dd-984e-e303c72205d1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:52.813373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7593-4db2-49dd-984e-e303c72205d1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:52.813504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e7593-4db2-49dd-984e-e303c72205d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:52.877721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-692e7593-4db2-49dd-984e-e303c72205d1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:52.878911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-692e7593-4db2-49dd-984e-e303c72205d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:52.887110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:52.977946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:53.015456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98754891-bc7b-41ca-bb4b-a906ddb669bb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:53.017068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-98754891-bc7b-41ca-bb4b-a906ddb669bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:53.121420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98754891-bc7b-41ca-bb4b-a906ddb669bb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:53.175703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:53.194270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:53.195260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:53.195787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:53.196377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:53.198111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:53.314333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:53.315777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001817ab-b363-4800-af97-fe24624a6f6d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:53.333546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 721/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:55.367104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d730b4-81b0-4d42-aa05-74659a158501 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:55.367683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d730b4-81b0-4d42-aa05-74659a158501 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:55.367683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d730b4-81b0-4d42-aa05-74659a158501 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:55.369093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:55.374930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785645-5aa0-4549-820c-b457997a9e04 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:55.375371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785645-5aa0-4549-820c-b457997a9e04 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:55.375497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96785645-5aa0-4549-820c-b457997a9e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:55.439715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-96785645-5aa0-4549-820c-b457997a9e04 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:55.440927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-96785645-5aa0-4549-820c-b457997a9e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:55.479922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:55.539532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:55.577383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d730b4-81b0-4d42-aa05-74659a158501 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:55.578914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-00d730b4-81b0-4d42-aa05-74659a158501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:55.681010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d730b4-81b0-4d42-aa05-74659a158501 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:55.717635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:55.754290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:55.754970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:55.755245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:55.755546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:55.757222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:55.865412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:55.867697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac0c37-5276-4c1d-ae1b-f792cead71e7 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:55.886324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 742/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:18:57.903913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e24e8-b7c6-4e7d-96de-b55f679e9951 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:57.904606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e24e8-b7c6-4e7d-96de-b55f679e9951 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:57.904606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e24e8-b7c6-4e7d-96de-b55f679e9951 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:57.905960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:18:57.911650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94429f04-25cf-422e-85f2-f91ff121e1f2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:57.912123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94429f04-25cf-422e-85f2-f91ff121e1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:57.912266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94429f04-25cf-422e-85f2-f91ff121e1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:57.985798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-94429f04-25cf-422e-85f2-f91ff121e1f2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:18:57.987004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-94429f04-25cf-422e-85f2-f91ff121e1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:58.005280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:58.082932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:18:58.119688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e24e8-b7c6-4e7d-96de-b55f679e9951 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:18:58.121543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7b5e24e8-b7c6-4e7d-96de-b55f679e9951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:58.223512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e24e8-b7c6-4e7d-96de-b55f679e9951 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:18:58.293661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:18:58.300513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:18:58.301166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:18:58.301448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:18:58.301722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:18:58.303373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:18:58.413158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:18:58.414763 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2d083-efeb-4c6e-b660-5c2822a27e72 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:18:58.433053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 724/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:00.458796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fd0f4-9255-47ea-87d1-63dfaf0236e8 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:00.459356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fd0f4-9255-47ea-87d1-63dfaf0236e8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:00.459356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922fd0f4-9255-47ea-87d1-63dfaf0236e8 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:00.460924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:00.466400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bf7f9-77be-460d-913b-5dacb45f5a62 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:00.466799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bf7f9-77be-460d-913b-5dacb45f5a62 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:00.466965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bf7f9-77be-460d-913b-5dacb45f5a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:00.532248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-672bf7f9-77be-460d-913b-5dacb45f5a62 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:00.533464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-672bf7f9-77be-460d-913b-5dacb45f5a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:00.580569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:00.630485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:00.669085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fd0f4-9255-47ea-87d1-63dfaf0236e8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:00.670650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-922fd0f4-9255-47ea-87d1-63dfaf0236e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:00.774477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922fd0f4-9255-47ea-87d1-63dfaf0236e8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:00.825109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:00.853206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:00.853890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:00.854175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:00.854452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:00.856354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:00.964761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:00.966217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abb598-6c63-498c-837d-0c8bcb9e754a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:00.984859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 745/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:10.628624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=22 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:10.982558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=66 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:20.535372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388f882-d292-4da4-bfe6-aebbd4e061e0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:20.535906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388f882-d292-4da4-bfe6-aebbd4e061e0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:20.535906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388f882-d292-4da4-bfe6-aebbd4e061e0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:20.537352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 726/1471] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:20.542680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf761b6-b03c-4257-8865-c580b0cee70d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:20.543086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf761b6-b03c-4257-8865-c580b0cee70d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:20.543229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf761b6-b03c-4257-8865-c580b0cee70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:20.607638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bcf761b6-b03c-4257-8865-c580b0cee70d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:20.608863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bcf761b6-b03c-4257-8865-c580b0cee70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:20.704325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 746/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:20.753418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401070d2-835d-4e6c-a15b-2f3b2a926d67 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:20.753708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401070d2-835d-4e6c-a15b-2f3b2a926d67 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:20.753986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401070d2-835d-4e6c-a15b-2f3b2a926d67 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:20.755369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 727/1473] 10.222.0.17 () {54 vars in 960 bytes} [Fri Jul 15 19:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:20.760802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b04e-7408-47b7-95f6-093a62daa47e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:20.761096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b04e-7408-47b7-95f6-093a62daa47e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:20.761377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e8b04e-7408-47b7-95f6-093a62daa47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:20.835519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c0e8b04e-7408-47b7-95f6-093a62daa47e None None] MFA Rules not processed for user `6802452aab2f43d4b965be689758b611`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:20.836677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0e8b04e-7408-47b7-95f6-093a62daa47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:20.858921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:20.914017 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 747/1474] 10.222.0.17 () {58 vars in 1057 bytes} [Fri Jul 15 19:19:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:20.921775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401070d2-835d-4e6c-a15b-2f3b2a926d67 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:20.923335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-401070d2-835d-4e6c-a15b-2f3b2a926d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:21.025372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401070d2-835d-4e6c-a15b-2f3b2a926d67 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:21.097904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:21.098481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:21.098766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:21.099045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:21.100723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:21.155085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:21.210841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:21.212308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eecfd8-f40c-4148-bb4e-91ca0a91a0b6 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:21.349938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 728/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:23.488807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf9d9-1ae7-4638-9d05-54e79759c713 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:23.489391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf9d9-1ae7-4638-9d05-54e79759c713 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:23.489391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bf9d9-1ae7-4638-9d05-54e79759c713 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:23.490779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 748/1476] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:23.496981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519023b-1659-45b8-a95c-b4687fc4a8ac None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:23.497401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519023b-1659-45b8-a95c-b4687fc4a8ac None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:23.497539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d519023b-1659-45b8-a95c-b4687fc4a8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:23.564248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d519023b-1659-45b8-a95c-b4687fc4a8ac None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:23.565440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d519023b-1659-45b8-a95c-b4687fc4a8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:23.658419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 729/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:23.682162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf9d9-1ae7-4638-9d05-54e79759c713 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:23.683845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-882bf9d9-1ae7-4638-9d05-54e79759c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:23.808642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bf9d9-1ae7-4638-9d05-54e79759c713 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:23.857400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:23.902881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:23.903541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:23.903821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:23.904114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:23.904854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:23.906654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:19:23.906654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:19:23.906654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:19:23.907812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf44b5cc-7d19-4dfa-bce6-6964344716a6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:23.926434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 749/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Fri Jul 15 19:19:23 2022] GET /identity/v3/projects => generated 2553 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:19:25.948645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4fce8-a797-42f5-af7c-bb40d512ccfa None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:25.949302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4fce8-a797-42f5-af7c-bb40d512ccfa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:25.949302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b4fce8-a797-42f5-af7c-bb40d512ccfa None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:25.950596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 730/1479] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:25.955792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e3a01-77cd-4c2d-823b-5165876ba1b8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:25.956232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e3a01-77cd-4c2d-823b-5165876ba1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:25.956381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e3a01-77cd-4c2d-823b-5165876ba1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:26.022238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e77e3a01-77cd-4c2d-823b-5165876ba1b8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:26.024187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e77e3a01-77cd-4c2d-823b-5165876ba1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:26.120726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 750/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:26.165359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4fce8-a797-42f5-af7c-bb40d512ccfa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:26.166907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f9b4fce8-a797-42f5-af7c-bb40d512ccfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:26.176708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:26.312739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b4fce8-a797-42f5-af7c-bb40d512ccfa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:26.393936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:26.427547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:26.428226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:26.428512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:26.428810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:26.430449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:26.517296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:26.518890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beae796-724b-4c5f-b340-7e2b2528ded1 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:26.537047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 731/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:28.706947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83726e4-37f7-427d-aff0-d98377d572ca None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:28.707541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83726e4-37f7-427d-aff0-d98377d572ca None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:28.707541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83726e4-37f7-427d-aff0-d98377d572ca None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:28.708940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 751/1482] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:28.714188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4008833-5b01-4085-8cc6-f3871f852a55 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:28.714578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4008833-5b01-4085-8cc6-f3871f852a55 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:28.714737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4008833-5b01-4085-8cc6-f3871f852a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:28.779297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b4008833-5b01-4085-8cc6-f3871f852a55 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:28.780572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b4008833-5b01-4085-8cc6-f3871f852a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:28.827017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:28.877525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 732/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:28.912642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83726e4-37f7-427d-aff0-d98377d572ca None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:28.914913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d83726e4-37f7-427d-aff0-d98377d572ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:28.938930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:29.023685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83726e4-37f7-427d-aff0-d98377d572ca None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:29.095949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:29.096623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:29.096908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:29.097187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:29.098865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:29.173292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:29.207299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:29.208792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c2e054-4706-49ac-99c0-c2ab9e31fe92 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:29.228054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 752/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:31.358832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cec7c-ad77-42f9-9a8a-8a27ffa4550b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:31.359417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cec7c-ad77-42f9-9a8a-8a27ffa4550b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:31.359417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32cec7c-ad77-42f9-9a8a-8a27ffa4550b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:31.360831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 733/1485] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:31.366126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f8ec-e2e3-45a1-ab84-8462b7b65636 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:31.367180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f8ec-e2e3-45a1-ab84-8462b7b65636 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:31.367180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517f8ec-e2e3-45a1-ab84-8462b7b65636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:31.434459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4517f8ec-e2e3-45a1-ab84-8462b7b65636 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:31.435679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4517f8ec-e2e3-45a1-ab84-8462b7b65636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:31.532607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 753/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:31.570476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cec7c-ad77-42f9-9a8a-8a27ffa4550b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:31.572040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e32cec7c-ad77-42f9-9a8a-8a27ffa4550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:31.673678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32cec7c-ad77-42f9-9a8a-8a27ffa4550b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:31.745683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:31.747783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:31.748495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:31.748969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:31.749391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:31.751228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:31.860604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:31.862035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ea363-4a01-4817-ac6d-5bfe305ed715 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:31.880225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 734/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:34.187812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e502a6-44cc-4717-8380-64d31b4fff99 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:34.188381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e502a6-44cc-4717-8380-64d31b4fff99 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:34.188381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e502a6-44cc-4717-8380-64d31b4fff99 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:34.189763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 754/1488] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:34.195321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff1b4d-2bfd-4e42-958b-e51078ab9b94 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:34.195733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff1b4d-2bfd-4e42-958b-e51078ab9b94 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:34.195862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff1b4d-2bfd-4e42-958b-e51078ab9b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:34.259396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-04ff1b4d-2bfd-4e42-958b-e51078ab9b94 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:34.260620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-04ff1b4d-2bfd-4e42-958b-e51078ab9b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:34.311797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:34.359532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 735/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:34.400765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e502a6-44cc-4717-8380-64d31b4fff99 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:34.402302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-70e502a6-44cc-4717-8380-64d31b4fff99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:34.437670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:34.506943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e502a6-44cc-4717-8380-64d31b4fff99 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:34.578978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:34.579630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:34.579912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:34.580224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:34.581872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:34.661413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:34.690304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:34.691762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33087519-981c-460b-bc6b-7661a2e06417 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:34.710662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 755/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:36.944235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc518f77-e91f-40ab-b34a-5575f82a9f3f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:36.945331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc518f77-e91f-40ab-b34a-5575f82a9f3f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:36.945443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc518f77-e91f-40ab-b34a-5575f82a9f3f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:36.946781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 736/1491] 10.222.0.17 () {54 vars in 926 bytes} [Fri Jul 15 19:19:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:19:36.951960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55246bd7-0ad2-4e18-a3f4-ef390c9c299a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:36.952399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55246bd7-0ad2-4e18-a3f4-ef390c9c299a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:36.952549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55246bd7-0ad2-4e18-a3f4-ef390c9c299a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:37.021139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-55246bd7-0ad2-4e18-a3f4-ef390c9c299a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:19:37.022337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-55246bd7-0ad2-4e18-a3f4-ef390c9c299a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:37.118170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 756/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Fri Jul 15 19:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:19:37.156465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc518f77-e91f-40ab-b34a-5575f82a9f3f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:19:37.158001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dc518f77-e91f-40ab-b34a-5575f82a9f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:37.261815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc518f77-e91f-40ab-b34a-5575f82a9f3f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:19:37.334134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:19:37.334776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:19:37.335574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:19:37.335849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:19:37.337510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:19:37.346318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:37.444109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:19:37.445525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f94231-d5f9-4a28-af5e-8be355b4de49 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:19:37.465609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 737/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:19:47.115726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=82 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:19:47.462894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=50 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:11.377560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:11.378190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:11.378190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:11.453235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:21:11.454447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:11.553445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 757/1494] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:21:11.562138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:11.562633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:11.562732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:11.639791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:21:11.640953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:11.740144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 738/1495] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:21:11.746393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:11.748121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:11.760681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca112c-9f33-425d-bb81-00f992587bcf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:11.821173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:11.855509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:11.856205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:11.856490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:11.856765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:11.857486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:21:11.857788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:11.858803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:11.858803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:21:11.858803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:21:11.859691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:11.870680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 758/1496] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:21:11.879182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:11.888268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6120503d-3a85-4fbd-8f12-e100505838e9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:11.948956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:11.983423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:11.984101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:11.984449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:11.984721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:11.985397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:11.987115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:11.987115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:21:11.987115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:21:11.988339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:12.041773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 739/1497] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:21:11 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:21:12.045969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:12.047649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:12.081000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829678a-35dc-485c-9d7f-10dc89974d6b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:12.174619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:12.175318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.175599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.175871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.176617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:12.177625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:12.177625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:21:12.177625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:21:12.178838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:12.220037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 759/1498] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:21:12 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:21:12.224058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:12.225592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:12.287942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044f2aac-f96a-4097-ad8f-422e7853c041 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:12.374522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:12.391040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:12.391764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.392044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.392345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.393158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:12.394173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:12.394173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:21:12.394173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:21:12.395070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:12.407548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 740/1499] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:21:12.412178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:12.413747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:12.425777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8093d-dbb9-4a31-9126-e60c17b7d76b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:12.450413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:12.522400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:12.522994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.523279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.523559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.525334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:12.526355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:12.526355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:21:12.526355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:21:12.527256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5688c44d-3bde-45b1-a1d4-6ca4dae5b71c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:12.540807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 760/1500] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:21:12.544355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:12.545911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:12.578337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa0c9b-9e10-47f4-9ea4-c2e3aa01ecb0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:12.672378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:12.673264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.673546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.673821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] PATH_INFO: `/v3/projects/ebbf363dc5364841bacc42a0f307f150/users/8453858db4eb4158856b4e38372ee4cf/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.695538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbf363dc5364841bacc42a0f307f150, user_id=8453858db4eb4158856b4e38372ee4cf, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:12.697205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:12.697205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:21:12.697205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:21:12.699178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bebda-e6fd-4b54-8c3d-1cc2dfe11e3b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:12.723643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 741/1501] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:21:12 2022] PUT /identity/v3/projects/ebbf363dc5364841bacc42a0f307f150/users/8453858db4eb4158856b4e38372ee4cf/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:21:12.737436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41fffa-c819-41e4-8604-26c39cfe8a44 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.737725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41fffa-c819-41e4-8604-26c39cfe8a44 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.738002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa41fffa-c819-41e4-8604-26c39cfe8a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.795517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fa41fffa-c819-41e4-8604-26c39cfe8a44 None None] MFA Rules not processed for user `8453858db4eb4158856b4e38372ee4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:21:12.796785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fa41fffa-c819-41e4-8604-26c39cfe8a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:12.874005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 761/1502] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:21:12.888176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecb42-fd1f-4273-a1ef-fb53d0c09c61 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.888470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecb42-fd1f-4273-a1ef-fb53d0c09c61 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.888749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cecb42-fd1f-4273-a1ef-fb53d0c09c61 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.890156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 742/1503] 10.222.0.17 () {54 vars in 963 bytes} [Fri Jul 15 19:21:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:21:12.894999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff7b54-2938-431e-aac0-bbad886dae2c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:12.895289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff7b54-2938-431e-aac0-bbad886dae2c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:12.895567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff7b54-2938-431e-aac0-bbad886dae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:12.925054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:12.969715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-80ff7b54-2938-431e-aac0-bbad886dae2c None None] MFA Rules not processed for user `43fda9627eb84093bb59ceb1a553baf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:21:12.970915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-80ff7b54-2938-431e-aac0-bbad886dae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:13.044931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 762/1504] 10.222.0.17 () {58 vars in 1060 bytes} [Fri Jul 15 19:21:12 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:21:13.052622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecb42-fd1f-4273-a1ef-fb53d0c09c61 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:13.054139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a5cecb42-fd1f-4273-a1ef-fb53d0c09c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:13.104773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:13.159161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cecb42-fd1f-4273-a1ef-fb53d0c09c61 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:13.231211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:13.231987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:13.232300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:13.232576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:13.234205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:13.333074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:13.334454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:13.338077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:13.352860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 743/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:21:13.395676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745b845-c06e-441a-979e-e677bec7ced1 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:13.395967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745b845-c06e-441a-979e-e677bec7ced1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:13.396301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d745b845-c06e-441a-979e-e677bec7ced1 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:13.397604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 763/1506] 10.222.0.17 () {52 vars in 829 bytes} [Fri Jul 15 19:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:21:13.401675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:13.402914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:13.509180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d3365-0e1f-434d-b6ee-f1144c5c1f9a service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:13.603104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:13.603891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:13.604201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:13.604475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] PATH_INFO: `/v3/users/8453858db4eb4158856b4e38372ee4cf` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:13.616865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8453858db4eb4158856b4e38372ee4cf)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:13.617951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:13.617951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:21:13.617951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:21:13.619215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:13.628807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:13.683228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8453858db4eb4158856b4e38372ee4cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:13.692975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8453858db4eb4158856b4e38372ee4cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:13.693412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8453858db4eb4158856b4e38372ee4cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:13.693962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8453858db4eb4158856b4e38372ee4cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:13.706569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8453858db4eb4158856b4e38372ee4cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:13.730237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 744/1507] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:21:13 2022] DELETE /identity/v3/users/8453858db4eb4158856b4e38372ee4cf => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:21:13.734813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:13.735292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:13.735397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:13.792133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:21:13.793380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:13.868724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:13.882730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 764/1508] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:21:13.892883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:13.894104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:13.993863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b5994-9ae2-4f6a-a97f-a8c9d4cf8ce0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:14.047886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:14.068913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:14.069576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:14.069888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:14.070163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:14.072392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:14.181314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:14.182683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7133ae-61ea-4364-8297-cf9cfa8d2643 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:14.201593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 745/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:21:14.627762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:21:14.629124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:21:14.641898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ff481-a064-4a6a-931a-f89c5b010450 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:21:14.737603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:21:14.738397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:21:14.738677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:21:14.738950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] PATH_INFO: `/v3/projects/ebbf363dc5364841bacc42a0f307f150` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:21:14.749468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbf363dc5364841bacc42a0f307f150)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:21:14.750510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:21:14.750510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:21:14.750510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:21:14.751755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:21:14.808943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbf363dc5364841bacc42a0f307f150', 'request_id': 'req-4f87a8ea-4222-422d-885d-bd052c04b750'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:14.818175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbf363dc5364841bacc42a0f307f150', 'request_id': 'req-4f87a8ea-4222-422d-885d-bd052c04b750'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:14.831560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbf363dc5364841bacc42a0f307f150', 'request_id': 'req-4f87a8ea-4222-422d-885d-bd052c04b750'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:14.855978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] The token cache is being invalidate because project ebbf363dc5364841bacc42a0f307f150 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:21:14.856305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:14.856740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4f87a8ea-4222-422d-885d-bd052c04b750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:21:14.858211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 765/1510] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:21:14 2022] DELETE /identity/v3/projects/ebbf363dc5364841bacc42a0f307f150 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:21:24.199309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=62 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:21:24.852498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:22:55.790951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7954af-f13a-406b-a202-458ce257efd7 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:55.791480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7954af-f13a-406b-a202-458ce257efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:55.791480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7954af-f13a-406b-a202-458ce257efd7 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:55.792960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:22:55.797822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b5025-5d94-475a-8fee-9a58846a6102 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:55.798264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b5025-5d94-475a-8fee-9a58846a6102 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:55.798409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b5025-5d94-475a-8fee-9a58846a6102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:55.877827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-de0b5025-5d94-475a-8fee-9a58846a6102 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:22:55.879044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-de0b5025-5d94-475a-8fee-9a58846a6102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:22:55.983994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:22:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:22:56.140210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7954af-f13a-406b-a202-458ce257efd7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:22:56.142782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dd7954af-f13a-406b-a202-458ce257efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:22:56.260190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7954af-f13a-406b-a202-458ce257efd7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:22:56.265527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c398cc4-1e92-4722-af0f-2bb0079b1b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:22:56.266166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c398cc4-1e92-4722-af0f-2bb0079b1b40 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:56.266449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c398cc4-1e92-4722-af0f-2bb0079b1b40 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:56.266726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c398cc4-1e92-4722-af0f-2bb0079b1b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:56.275315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c398cc4-1e92-4722-af0f-2bb0079b1b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:22:56.276967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c398cc4-1e92-4722-af0f-2bb0079b1b40 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:22:56.286323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 747/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:22:57.139811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552ad34-a5fc-4e2c-8b7b-57ca30947129 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:57.140413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552ad34-a5fc-4e2c-8b7b-57ca30947129 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:57.140413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552ad34-a5fc-4e2c-8b7b-57ca30947129 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:57.141786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:22:57.146576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020f79-6d0a-46ef-aae6-2dd69a02b5d6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:57.146998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020f79-6d0a-46ef-aae6-2dd69a02b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:57.147139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020f79-6d0a-46ef-aae6-2dd69a02b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:57.166679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9a020f79-6d0a-46ef-aae6-2dd69a02b5d6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:22:57.167870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9a020f79-6d0a-46ef-aae6-2dd69a02b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:22:57.177831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:22:57.328759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552ad34-a5fc-4e2c-8b7b-57ca30947129 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:22:57.343327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552ad34-a5fc-4e2c-8b7b-57ca30947129 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:22:57.348682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b583f0f-777d-45fe-85e4-84107933599f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:22:57.349316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b583f0f-777d-45fe-85e4-84107933599f service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:57.349600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b583f0f-777d-45fe-85e4-84107933599f service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:57.349873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b583f0f-777d-45fe-85e4-84107933599f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:57.358524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b583f0f-777d-45fe-85e4-84107933599f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:22:57.360193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b583f0f-777d-45fe-85e4-84107933599f service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:22:57.370033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 768/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:22:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:22:57.626462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc63187-3669-4619-82a2-11e28dded5bf None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:57.626744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc63187-3669-4619-82a2-11e28dded5bf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:57.627017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc63187-3669-4619-82a2-11e28dded5bf None None] PATH_INFO: `/v3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:57.628432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 749/1517] 10.222.0.26 () {54 vars in 925 bytes} [Fri Jul 15 19:22:57 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:22:57.635259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08be160-9f6d-4d48-b2ae-1cb41bd0fbff None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:22:57.635543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08be160-9f6d-4d48-b2ae-1cb41bd0fbff None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:22:57.635818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08be160-9f6d-4d48-b2ae-1cb41bd0fbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:22:57.811124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e08be160-9f6d-4d48-b2ae-1cb41bd0fbff None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:22:57.812341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e08be160-9f6d-4d48-b2ae-1cb41bd0fbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:22:57.829162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:22:57.882623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 769/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Jul 15 19:22:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:23:06.749749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433691d-fc01-4d22-8629-f4127eff1111 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:06.750329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433691d-fc01-4d22-8629-f4127eff1111 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:06.750509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433691d-fc01-4d22-8629-f4127eff1111 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:06.751921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 750/1519] 10.222.0.17 () {54 vars in 960 bytes} [Fri Jul 15 19:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:06.756895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc7a0f-28b0-4afb-a62c-c313866357a0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:06.757326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc7a0f-28b0-4afb-a62c-c313866357a0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:06.757480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc7a0f-28b0-4afb-a62c-c313866357a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:06.776349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c2cc7a0f-28b0-4afb-a62c-c313866357a0 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:06.777667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c2cc7a0f-28b0-4afb-a62c-c313866357a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:06.786131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 770/1520] 10.222.0.17 () {58 vars in 1057 bytes} [Fri Jul 15 19:23:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:23:06.794383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433691d-fc01-4d22-8629-f4127eff1111 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:06.807451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433691d-fc01-4d22-8629-f4127eff1111 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:06.812272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109bc3f-6034-407b-92d3-68c5af90d00f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:06.812902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109bc3f-6034-407b-92d3-68c5af90d00f service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:06.813185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109bc3f-6034-407b-92d3-68c5af90d00f service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:06.813461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109bc3f-6034-407b-92d3-68c5af90d00f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:06.821548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109bc3f-6034-407b-92d3-68c5af90d00f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:06.823288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109bc3f-6034-407b-92d3-68c5af90d00f service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:06.832834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 751/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:23:08.615452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c743-f676-488a-8ef4-ec1cfe099fcc None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:08.615998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c743-f676-488a-8ef4-ec1cfe099fcc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:08.615998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12c743-f676-488a-8ef4-ec1cfe099fcc None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:08.617406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:08.622120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e353f46-c5c8-4b0a-9ed0-ba1cdd116a2b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:08.622550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e353f46-c5c8-4b0a-9ed0-ba1cdd116a2b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:08.622669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e353f46-c5c8-4b0a-9ed0-ba1cdd116a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:08.641363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1e353f46-c5c8-4b0a-9ed0-ba1cdd116a2b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:08.642556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1e353f46-c5c8-4b0a-9ed0-ba1cdd116a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:08.650949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:23:08.803914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c743-f676-488a-8ef4-ec1cfe099fcc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:08.818395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12c743-f676-488a-8ef4-ec1cfe099fcc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:08.824692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c0365-c93b-415b-9aa5-2901651c2821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:08.825334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c0365-c93b-415b-9aa5-2901651c2821 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:08.825619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c0365-c93b-415b-9aa5-2901651c2821 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:08.825897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c0365-c93b-415b-9aa5-2901651c2821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:08.834026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c0365-c93b-415b-9aa5-2901651c2821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:08.835602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c0365-c93b-415b-9aa5-2901651c2821 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:08.846284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 772/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:23:09.667865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8cf6-a51a-4434-96a6-b8a1feb7ef72 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:09.668502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8cf6-a51a-4434-96a6-b8a1feb7ef72 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:09.668502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a8cf6-a51a-4434-96a6-b8a1feb7ef72 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:09.669838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jul 15 19:23:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:09.674810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25eb72-f2ab-4046-ae98-3d2d2df59218 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:09.675270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25eb72-f2ab-4046-ae98-3d2d2df59218 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:09.675381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25eb72-f2ab-4046-ae98-3d2d2df59218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:09.693780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fa25eb72-f2ab-4046-ae98-3d2d2df59218 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:09.694970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fa25eb72-f2ab-4046-ae98-3d2d2df59218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:09.703213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jul 15 19:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:23:09.860725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a8cf6-a51a-4434-96a6-b8a1feb7ef72 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:09.875821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a8cf6-a51a-4434-96a6-b8a1feb7ef72 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:09.881430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941b916-6a63-4c02-92e5-f650cde14556 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:09.882073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b916-6a63-4c02-92e5-f650cde14556 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:09.882363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b916-6a63-4c02-92e5-f650cde14556 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:09.882646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941b916-6a63-4c02-92e5-f650cde14556 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:09.891135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941b916-6a63-4c02-92e5-f650cde14556 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:09.892785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941b916-6a63-4c02-92e5-f650cde14556 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:09.902786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 754/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:23:13.821652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc390d98-2022-49d6-9d7c-6ff77c1c2344 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:13.822900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc390d98-2022-49d6-9d7c-6ff77c1c2344 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:13.823567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc390d98-2022-49d6-9d7c-6ff77c1c2344 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:13.824988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:23:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:13.830271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610aff-831a-4bc8-84c9-f469922cebda None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:13.830686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610aff-831a-4bc8-84c9-f469922cebda None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:13.830824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610aff-831a-4bc8-84c9-f469922cebda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:13.848598 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a3610aff-831a-4bc8-84c9-f469922cebda None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:13.849787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a3610aff-831a-4bc8-84c9-f469922cebda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:13.858298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:23:13.895528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc390d98-2022-49d6-9d7c-6ff77c1c2344 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:13.897236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bc390d98-2022-49d6-9d7c-6ff77c1c2344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:13.975029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc390d98-2022-49d6-9d7c-6ff77c1c2344 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:13.979050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685a9ba-1a70-44a9-9cca-6a3d8549bbaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:13.979686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685a9ba-1a70-44a9-9cca-6a3d8549bbaf service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:13.979966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685a9ba-1a70-44a9-9cca-6a3d8549bbaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:13.980269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685a9ba-1a70-44a9-9cca-6a3d8549bbaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:13.988971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685a9ba-1a70-44a9-9cca-6a3d8549bbaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:13.990399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685a9ba-1a70-44a9-9cca-6a3d8549bbaf service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:14.003422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 775/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:23:16.040576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1e10-9c0b-403d-b54b-7da1302e02f7 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:16.041106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1e10-9c0b-403d-b54b-7da1302e02f7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:16.041106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1e10-9c0b-403d-b54b-7da1302e02f7 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:16.042440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:16.047666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b8d06-9e85-46db-8247-abe645aa85b2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:16.048063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b8d06-9e85-46db-8247-abe645aa85b2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:16.048406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b8d06-9e85-46db-8247-abe645aa85b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:16.065875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e09b8d06-9e85-46db-8247-abe645aa85b2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:16.067052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e09b8d06-9e85-46db-8247-abe645aa85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:16.075493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:23:16.111680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b1e10-9c0b-403d-b54b-7da1302e02f7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:16.113406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a52b1e10-9c0b-403d-b54b-7da1302e02f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:16.127324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b1e10-9c0b-403d-b54b-7da1302e02f7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:16.130800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc2797-9ce7-425b-b771-ba5f77392ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:16.131436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc2797-9ce7-425b-b771-ba5f77392ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:16.131721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc2797-9ce7-425b-b771-ba5f77392ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:16.131997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc2797-9ce7-425b-b771-ba5f77392ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:16.141007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc2797-9ce7-425b-b771-ba5f77392ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:16.142436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc2797-9ce7-425b-b771-ba5f77392ee7 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:16.151793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 757/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:23:19.801266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843687-193c-4a00-aebe-f76ba47bf25b None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:19.801733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843687-193c-4a00-aebe-f76ba47bf25b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:19.801827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76843687-193c-4a00-aebe-f76ba47bf25b None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:19.803184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:19.808589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed264ebc-7e42-423d-964c-694582f60dc7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:19.808997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed264ebc-7e42-423d-964c-694582f60dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:19.809154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed264ebc-7e42-423d-964c-694582f60dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:19.826846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ed264ebc-7e42-423d-964c-694582f60dc7 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:19.828036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ed264ebc-7e42-423d-964c-694582f60dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:19.835789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:23:20.138702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76843687-193c-4a00-aebe-f76ba47bf25b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:20.140450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-76843687-193c-4a00-aebe-f76ba47bf25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:20.206035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:23:20.219235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76843687-193c-4a00-aebe-f76ba47bf25b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:20.222784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801eda51-b8a1-4a7f-82ec-1d2da0a07d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:20.223428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eda51-b8a1-4a7f-82ec-1d2da0a07d04 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:20.223714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eda51-b8a1-4a7f-82ec-1d2da0a07d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:20.223989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801eda51-b8a1-4a7f-82ec-1d2da0a07d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:20.233565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eda51-b8a1-4a7f-82ec-1d2da0a07d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:20.235009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801eda51-b8a1-4a7f-82ec-1d2da0a07d04 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:20.244561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 778/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:23:22.269069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d40bc-0f9a-4404-a335-8520f20afa3e None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:22.269604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d40bc-0f9a-4404-a335-8520f20afa3e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:22.269604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d40bc-0f9a-4404-a335-8520f20afa3e None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:22.271017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jul 15 19:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:23:22.276397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbdc8d-9a72-44b1-969f-cfcbc21c0ba1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:22.276839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbdc8d-9a72-44b1-969f-cfcbc21c0ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:22.276937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fbdc8d-9a72-44b1-969f-cfcbc21c0ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:22.295316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-59fbdc8d-9a72-44b1-969f-cfcbc21c0ba1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:23:22.296552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-59fbdc8d-9a72-44b1-969f-cfcbc21c0ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:22.304576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jul 15 19:23:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:23:22.611473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d40bc-0f9a-4404-a335-8520f20afa3e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:23:22.613164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2a8d40bc-0f9a-4404-a335-8520f20afa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:23:22.626447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d40bc-0f9a-4404-a335-8520f20afa3e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:23:22.629941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae0ceb-3bed-4f23-874a-4e365ea65e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:23:22.630578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae0ceb-3bed-4f23-874a-4e365ea65e50 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:23:22.630861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae0ceb-3bed-4f23-874a-4e365ea65e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:23:22.631137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae0ceb-3bed-4f23-874a-4e365ea65e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:23:22.640248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae0ceb-3bed-4f23-874a-4e365ea65e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:23:22.641680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddae0ceb-3bed-4f23-874a-4e365ea65e50 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:23:22.651573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 760/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:23:32.289404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=11 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:23:32.646959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:52.076016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4205662-23f6-4839-8444-b174d51585b2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.076667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4205662-23f6-4839-8444-b174d51585b2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.076667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4205662-23f6-4839-8444-b174d51585b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.100712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a4205662-23f6-4839-8444-b174d51585b2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.103321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a4205662-23f6-4839-8444-b174d51585b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.121246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 780/1540] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.133080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.134068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.134564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.143186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa394ec3-1deb-4cc2-adbb-76cc2c48db2e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.143478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa394ec3-1deb-4cc2-adbb-76cc2c48db2e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.143753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa394ec3-1deb-4cc2-adbb-76cc2c48db2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.164761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.166959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.177098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fa394ec3-1deb-4cc2-adbb-76cc2c48db2e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.178661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fa394ec3-1deb-4cc2-adbb-76cc2c48db2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.183267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 761/1541] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.188149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.195758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 781/1542] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:24:52.198978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826e6757-dbc5-4e43-9e61-62e87aff025c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.203956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d01bd3-a666-4b26-be68-40014345c886 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.204275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d01bd3-a666-4b26-be68-40014345c886 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.204556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d01bd3-a666-4b26-be68-40014345c886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.213485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.214414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.214698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.214976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.215719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.216030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.218042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:24:52.218042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:24:52.218042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:24:52.218962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.236742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-51d01bd3-a666-4b26-be68-40014345c886 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.237987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 762/1543] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:24:52.240251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-51d01bd3-a666-4b26-be68-40014345c886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.246697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.253960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 782/1544] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:24:52.255435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.255720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.255994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.259638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377eeba1-d255-4bf1-aca9-37449aeb3c49 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.276714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e067320f-b225-44bb-92f5-4cfb777f1ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.277394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067320f-b225-44bb-92f5-4cfb777f1ddd admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.277686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067320f-b225-44bb-92f5-4cfb777f1ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.277965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e067320f-b225-44bb-92f5-4cfb777f1ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.279193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067320f-b225-44bb-92f5-4cfb777f1ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.281174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e067320f-b225-44bb-92f5-4cfb777f1ddd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.296157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.297365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.323089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 783/1545] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.323953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.340938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c029-0dfb-4b09-a501-7bdf829d215e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.349320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.350210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.350497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.350789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.351661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.351963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.353694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.372065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 763/1546] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:24:52.378478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5545bfe-ffef-4345-ab13-42fcacd70940 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.379069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 784/1547] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:52.384807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5545bfe-ffef-4345-ab13-42fcacd70940 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.385013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.385719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5545bfe-ffef-4345-ab13-42fcacd70940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.406728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa8adf-8e20-45ec-aca6-4672fe360e0d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.420733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c5545bfe-ffef-4345-ab13-42fcacd70940 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.422515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c5545bfe-ffef-4345-ab13-42fcacd70940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.482024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa659c-6527-4c9d-93b4-976a05ab7a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.483069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa659c-6527-4c9d-93b4-976a05ab7a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.483557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa659c-6527-4c9d-93b4-976a05ab7a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.484059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa659c-6527-4c9d-93b4-976a05ab7a52 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.484786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa659c-6527-4c9d-93b4-976a05ab7a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.486733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa659c-6527-4c9d-93b4-976a05ab7a52 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.502000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 764/1548] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.503584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bdf89-04ff-4b37-b7d1-d6ae776a5558 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.503938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bdf89-04ff-4b37-b7d1-d6ae776a5558 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.504322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386bdf89-04ff-4b37-b7d1-d6ae776a5558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.516536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 785/1549] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:52.516536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.516536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.516536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.524045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-386bdf89-04ff-4b37-b7d1-d6ae776a5558 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.525347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-386bdf89-04ff-4b37-b7d1-d6ae776a5558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.534848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 765/1550] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.536746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e0bec-d3a4-4fc5-99f3-7b5b54a833d4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.537095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.537464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e0bec-d3a4-4fc5-99f3-7b5b54a833d4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.537981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e0bec-d3a4-4fc5-99f3-7b5b54a833d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.538292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.546623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 786/1551] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.547515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.559016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041d0b6-fa3d-4b29-a6ee-3483a209a688 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.563027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-469e0bec-d3a4-4fc5-99f3-7b5b54a833d4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.563921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.564495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-469e0bec-d3a4-4fc5-99f3-7b5b54a833d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.564812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.565168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.565517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.566279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.567408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:24:52.567408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:24:52.567408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:24:52.568782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.573397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 766/1552] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.574954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.575251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.575650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.598028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.599158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.609570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 767/1553] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.616313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.620765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082c88f4-4e3d-46b8-8a2f-a6fd7beb9696 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.625626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 787/1554] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:52.626731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.627588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.628547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.628970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.629321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.630487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.630805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.632954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751cee6-d72b-4ba0-a599-e85fc02df87a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.647458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 768/1555] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:52.649502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef84c-555f-4678-9c76-b2abcf2943c6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.649796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef84c-555f-4678-9c76-b2abcf2943c6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.650074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8ef84c-555f-4678-9c76-b2abcf2943c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.669443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ec8ef84c-555f-4678-9c76-b2abcf2943c6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.670660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ec8ef84c-555f-4678-9c76-b2abcf2943c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.695117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf94a7-4732-44ec-bf63-84246c491144 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.699770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd756d-ca7f-41e2-a828-ad5072c2cdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.700464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd756d-ca7f-41e2-a828-ad5072c2cdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.700750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd756d-ca7f-41e2-a828-ad5072c2cdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.701026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd756d-ca7f-41e2-a828-ad5072c2cdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.701756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd756d-ca7f-41e2-a828-ad5072c2cdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.703353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd756d-ca7f-41e2-a828-ad5072c2cdf6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.714295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 788/1556] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:52.715532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd681-3959-4bff-a434-6491999804fa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.715825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd681-3959-4bff-a434-6491999804fa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.716127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cd681-3959-4bff-a434-6491999804fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.735218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 769/1557] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.735426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bf7cd681-3959-4bff-a434-6491999804fa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.736627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bf7cd681-3959-4bff-a434-6491999804fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.737152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.737601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.738026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.747120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 789/1558] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.749671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7440-c197-4b47-9fa7-d3337cfd03f3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.750035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7440-c197-4b47-9fa7-d3337cfd03f3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.750326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa7440-c197-4b47-9fa7-d3337cfd03f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.761622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.763268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.768034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-58aa7440-c197-4b47-9fa7-d3337cfd03f3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.769227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-58aa7440-c197-4b47-9fa7-d3337cfd03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.771401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 770/1559] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.771987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.781840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d802483-6a95-450c-a269-9fba96b73ffc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.782274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 790/1560] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.783634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.783875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.784187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.786516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.787206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.787489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.787765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.788481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.789655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:24:52.789655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:24:52.789655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:24:52.790876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.792346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:52.800628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:52.803406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:52.805246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:52.813425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 771/1561] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:52.814360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 791/1562] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:52.814607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.815151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.826463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d502-0d7c-443d-8701-e0af767e6cea None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.826865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa94c1c-af29-489e-b10e-cd1728e768ff admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.831137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.831347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.831792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.831993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.832120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.832327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.832423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.832608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.833238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.833348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.834837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.835197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.844800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 792/1563] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:52.845630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.856178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68fb83-aa12-432a-837a-5fb1c0bd1007 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.861569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.862206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.862488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.862768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.863490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.863818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.868852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 772/1564] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:52.869016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.869148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.885903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 793/1565] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:52.885903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.890638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905d0c96-ef32-4b7b-821a-b45383930b74 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.935947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad1595-fd44-4f0d-b244-bed921b00beb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.940928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.941584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.941868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.942142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.942865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.943167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.945412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.948917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.949580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.949864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.950143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.950864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.951173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.952809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.957759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 773/1566] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:52.961089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.971228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c7af1-5109-4319-9ac0-68cb681eb930 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.971934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 794/1567] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:52.972894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:52.975770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.977078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.977361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.977645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.978354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:52.978659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.980273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4df45e-4402-4dab-a8df-d8dd58ca53a7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.983369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d04ca-eaea-46a8-8a18-33d78579e69d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:52.987957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:52.988634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.988921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.989200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:52.989916 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:52.991539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:52.992798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 774/1568] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:52.996935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:52.997233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:52.997513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.003064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 795/1569] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.003953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.014268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea0aee-453b-4325-8a7c-0c022cbee92c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.017902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:53.018839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.019090 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:53.019705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.019986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.020289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] PATH_INFO: `/v3/projects/e39f25de38b1465784de70ebf4e930a9/users/837f13aef58b455e95372c6f106d5807/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.027952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 775/1570] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:53.028753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.036370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e39f25de38b1465784de70ebf4e930a9, user_id=837f13aef58b455e95372c6f106d5807, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.037415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:24:53.037415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:24:53.037415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:24:53.038409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82308e7e-f730-4fba-84ef-21bf5244936d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.039595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.043139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.043915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.044100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.044388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.045168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.047248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.053344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 796/1571] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:53 2022] PUT /identity/v3/projects/e39f25de38b1465784de70ebf4e930a9/users/837f13aef58b455e95372c6f106d5807/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:53.054680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.067946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 776/1572] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/users => generated 289 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.068777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.081462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda983ea-951b-45ba-a7a0-eb2f03657ecf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.121648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea50104-5894-4dd6-8dec-12cb954345cd admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.126781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.127503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.127815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.128104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.128803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.130716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.139535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.140284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.140568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.140839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.141553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.143450 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.156223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 797/1573] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.157169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.173909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 777/1574] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/projects => generated 393 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.177618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:53.177809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.192319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:53.221247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a1282-02a2-4267-9aaf-896f33f69941 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.241592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86d0989-a233-42f6-b9b8-fd6ec069e0dd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.246805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.247454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.247874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.248188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.249140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.251138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.280651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 778/1575] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.280651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.280935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.280935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.280935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.280935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.281251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.282574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.309524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 798/1576] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/projects => generated 387 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.310349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.349637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b1976-285f-4bcc-a75e-82013c034753 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.373238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b794dc-3099-4280-93af-a226d8affc6f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.377983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.378638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.378915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.379184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.379912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:24:53.380247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.381845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c326bce-193b-4448-bb7d-38054ffa8f5e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.393480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 799/1577] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:24:53.394812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.395272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.395387 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.408373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.409048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.409335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.409616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.410347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.411959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.412439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:53.414239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:53.422130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 800/1578] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:53.428832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.429526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 779/1579] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.429526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.432466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7771025-0804-4afe-aa22-f4a72cdf0777 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.435122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b298f10b-4c1c-4b0e-8635-ab7fcb9fb3fb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.437008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.437668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.437946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.438217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.438937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.439597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.440293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.440575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.440682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.440854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.441525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.444002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.450699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 801/1580] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.451630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.463002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8427b63-9354-4bae-bca5-ff36d205d0df admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.465322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 780/1581] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.466069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.470635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.471628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.472129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.472411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.473134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.475204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.475327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81254fa-644c-4be6-a52f-a3b446e4774e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.481301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.482218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.482509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.482784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.483459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.485391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.497487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 802/1582] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/users => generated 297 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.498245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.509096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51390441-f569-4a99-b7ed-ab34daccfada admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.512541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 781/1583] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/users => generated 283 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.512541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.516719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.517393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.517673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.517950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.518633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.520233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3e60b-4b62-4d10-b20e-3da2c6a45219 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.520627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.525300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.525932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.526214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.526490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.527160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.528737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:53.529069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.533195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:53.543675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 803/1584] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.544461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.554636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3f8f4c-fb64-4317-96c8-8d3ef5b9914b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.559193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.560059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.560190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 782/1585] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.560932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.563367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.563657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] PATH_INFO: `/v3/projects/cb15403b3b5d462ab74cee11b5521792/users/e4d49f9f74ce4feda54e3eebcc2bbdf0/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.579317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb15403b3b5d462ab74cee11b5521792, user_id=e4d49f9f74ce4feda54e3eebcc2bbdf0, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.582025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.595693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 804/1586] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:53 2022] PUT /identity/v3/projects/cb15403b3b5d462ab74cee11b5521792/users/e4d49f9f74ce4feda54e3eebcc2bbdf0/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:53.596523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.628149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10d066-7e5c-4add-be34-e65165f6e688 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.659672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03c9d3-a2f5-45a8-aa42-dd9c2d30b1bb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.664662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.665399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.665688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.665966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.666698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.668394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.678386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 805/1587] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.678603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.679282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.679410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.679591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.679848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.689353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.690820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.691446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0535ad9-12ac-4a3e-ab82-1906c5c00c9f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.697245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.697888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.698170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.698451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.699165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.700812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.700937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 783/1588] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:53.707426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.711605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 806/1589] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.712487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.717663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a925c6-fa03-4939-bca1-6ac1a96fddb5 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.722405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30962a-adf5-4cbb-8bc4-803c675d82ad admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.726579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.727529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.728026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.728367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.729090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.729332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.729871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.730340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.730654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.730997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.731441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.733460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.747185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 784/1590] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.747185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.749640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 807/1591] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.750497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.756948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cfec1d-c19e-4b28-976d-96d3124ff307 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.761767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dbde2-6319-428d-81ce-6488c9dd7d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.762468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbde2-6319-428d-81ce-6488c9dd7d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.762729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf36d8-3e0c-40c2-a6fb-b88537f203da admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.762863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbde2-6319-428d-81ce-6488c9dd7d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.763014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dbde2-6319-428d-81ce-6488c9dd7d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.763747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dbde2-6319-428d-81ce-6488c9dd7d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.765389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dbde2-6319-428d-81ce-6488c9dd7d14 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.767443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.768224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.768518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.768797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.769493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.771650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.778800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 785/1592] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.780457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.780970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.781515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.796129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 808/1593] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/users => generated 286 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:53.797042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.808373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6540d-41ab-49e0-b0a0-7ad7fc71a108 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.813690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.814594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.814997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.815332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] PATH_INFO: `/v3/projects/b2599f1a3d5d40a4add00e3343d1c4b4/users/d06d1b5c0eff4645bf7fb43f8921bb17/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.815858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] MFA Rules not processed for user `e4d49f9f74ce4feda54e3eebcc2bbdf0`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:53.817162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:53.833589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2599f1a3d5d40a4add00e3343d1c4b4, user_id=d06d1b5c0eff4645bf7fb43f8921bb17, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.836531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.853949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 809/1594] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:53 2022] PUT /identity/v3/projects/b2599f1a3d5d40a4add00e3343d1c4b4/users/d06d1b5c0eff4645bf7fb43f8921bb17/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:53.854879 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.905677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 786/1595] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:53.906899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.925059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b69a2c0-0f35-48c2-bfd8-4b7d81cdd046 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.935708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:53.951785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c4338-8652-490a-b10c-f31d0a2840bb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.959190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.960291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.961396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.961854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.963424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:53.965126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:53.966752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:53.978270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 810/1596] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:53.979433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:53.992702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2fb56-17f3-4d31-9a10-bd2492e04f00 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:53.997902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.998171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:53.998846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:53.999133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:53.999411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:53.999950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.000177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.000322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.000488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.001374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.001849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.003043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.014508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 787/1597] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:54.015505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.022060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 811/1598] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:54.022779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.028489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae1868-51a7-485d-8f62-8cc66c44e491 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.033517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac857d-d7d4-455c-91d1-954ee041f275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.034188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac857d-d7d4-455c-91d1-954ee041f275 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.034474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac857d-d7d4-455c-91d1-954ee041f275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.034751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac857d-d7d4-455c-91d1-954ee041f275 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.035193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decf570f-ed77-4a0d-bf91-9dc1cb631f36 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.035481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac857d-d7d4-455c-91d1-954ee041f275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.037237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ac857d-d7d4-455c-91d1-954ee041f275 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.042408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215c69d-4f82-44ac-aceb-52170e932a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.043823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215c69d-4f82-44ac-aceb-52170e932a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.044203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215c69d-4f82-44ac-aceb-52170e932a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.044481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215c69d-4f82-44ac-aceb-52170e932a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.045542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215c69d-4f82-44ac-aceb-52170e932a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.047169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215c69d-4f82-44ac-aceb-52170e932a34 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.048469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 788/1599] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:54.049965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.050273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.050528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.063503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 812/1600] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:54.065771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb4b2-fd53-402a-baf5-4dafc17f0603 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.066459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb4b2-fd53-402a-baf5-4dafc17f0603 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.067121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffbb4b2-fd53-402a-baf5-4dafc17f0603 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.068532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 813/1601] 10.222.0.17 () {54 vars in 961 bytes} [Fri Jul 15 19:24:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:24:54.069334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb4b2-fd53-402a-baf5-4dafc17f0603 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.077524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:54.078756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:54.088246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 789/1602] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:54.088744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffbb4b2-fd53-402a-baf5-4dafc17f0603 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.089196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.099102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69d364-e580-49a1-b986-9f5139eebb0b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.099967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.101007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.102062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.102639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] PATH_INFO: `/v3/projects/ebcd8788770344d290ebf376ec33e7ad/users/bf32cd62eec04de892c3b27fbdd882a8/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.104716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.105658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.105938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.106217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] PATH_INFO: `/v3/projects/4b566a1f9c8e466585ece82f307f41cb/users/025ea412ccfa455aa84d051244902cd1/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.120198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebcd8788770344d290ebf376ec33e7ad, user_id=bf32cd62eec04de892c3b27fbdd882a8, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.121804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b566a1f9c8e466585ece82f307f41cb, user_id=025ea412ccfa455aa84d051244902cd1, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.124812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.125366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.146239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 790/1604] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:54 2022] PUT /identity/v3/projects/4b566a1f9c8e466585ece82f307f41cb/users/025ea412ccfa455aa84d051244902cd1/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:54.147475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.150253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 814/1604] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:54 2022] PUT /identity/v3/projects/ebcd8788770344d290ebf376ec33e7ad/users/bf32cd62eec04de892c3b27fbdd882a8/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:54.151306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.230212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76168580-566c-45d8-a768-8b4810d10fbf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.232024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331aaf69-586b-4ff2-9d78-9726118ebf5c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.236306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de8d7-42a2-4c83-99c6-971ba3bfb4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.237173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de8d7-42a2-4c83-99c6-971ba3bfb4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.237463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de8d7-42a2-4c83-99c6-971ba3bfb4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.237568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.237796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de8d7-42a2-4c83-99c6-971ba3bfb4b9 admin admin] PATH_INFO: `/v3/projects/4069de5b46764795816dcf7d6f525dcf/users/14ba4ed80c014715a763d9b542efe534/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.238425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.238710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.238991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] PATH_INFO: `/v3/projects/442be57bdd524fa092e7ebc12bd007a1/users/01120eedaa844f24a66024df8d3b2271/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.254694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2de8d7-42a2-4c83-99c6-971ba3bfb4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4069de5b46764795816dcf7d6f525dcf, user_id=14ba4ed80c014715a763d9b542efe534, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.256541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=442be57bdd524fa092e7ebc12bd007a1, user_id=01120eedaa844f24a66024df8d3b2271, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.259675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.262172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2de8d7-42a2-4c83-99c6-971ba3bfb4b9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.278697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 815/1605] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:54 2022] PUT /identity/v3/projects/442be57bdd524fa092e7ebc12bd007a1/users/01120eedaa844f24a66024df8d3b2271/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:54.279392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.282860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 791/1606] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:54 2022] PUT /identity/v3/projects/4069de5b46764795816dcf7d6f525dcf/users/14ba4ed80c014715a763d9b542efe534/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:54.284025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.284351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.284635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.328809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:54.330044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:54.361413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf89f953-03c8-4fd0-ae61-240391e05cdf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.368564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:54.407048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:54.423294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 792/1607] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Jul 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:54.424336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.431063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfc2cf-d0d4-4a28-8c9b-00ced37f79dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.432338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfc2cf-d0d4-4a28-8c9b-00ced37f79dc admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.432807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfc2cf-d0d4-4a28-8c9b-00ced37f79dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.433358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfc2cf-d0d4-4a28-8c9b-00ced37f79dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.434249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfc2cf-d0d4-4a28-8c9b-00ced37f79dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.436180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfc2cf-d0d4-4a28-8c9b-00ced37f79dc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.452170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 816/1608] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:54.454270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5443d-67d3-452e-9a0e-d7cbb6cff4ed None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.454849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5443d-67d3-452e-9a0e-d7cbb6cff4ed None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.455246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5443d-67d3-452e-9a0e-d7cbb6cff4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.478959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-54c5443d-67d3-452e-9a0e-d7cbb6cff4ed None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:54.480478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-54c5443d-67d3-452e-9a0e-d7cbb6cff4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:54.490433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 817/1609] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:54.491866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a3926-3a2f-4244-baaf-eabb0f2af879 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.492183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a3926-3a2f-4244-baaf-eabb0f2af879 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.492467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44a3926-3a2f-4244-baaf-eabb0f2af879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.508354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfbcd9c-8b5f-4483-ba46-f7e7aeec94ca None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.512251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53239919-83b5-42fb-bcdf-120dc269ad3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.512914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53239919-83b5-42fb-bcdf-120dc269ad3b service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.513325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53239919-83b5-42fb-bcdf-120dc269ad3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.513605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53239919-83b5-42fb-bcdf-120dc269ad3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.527117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53239919-83b5-42fb-bcdf-120dc269ad3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.528924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53239919-83b5-42fb-bcdf-120dc269ad3b service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.533220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e44a3926-3a2f-4244-baaf-eabb0f2af879 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:54.535665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e44a3926-3a2f-4244-baaf-eabb0f2af879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:54.540372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 793/1610] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:54.542056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.542411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.542848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.549255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 818/1611] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:54.558262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.558560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.559315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.563603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:54.565017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:54.574244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 794/1612] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:54.575891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.594826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a024c-c1d9-4a6a-8efe-dbb3ed03554f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.602432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.602789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] MFA Rules not processed for user `14ba4ed80c014715a763d9b542efe534`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:54.603339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.603722 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.604107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.604226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:54.676850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.678508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.688164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 819/1613] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:54.688164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.694402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 795/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:54.695663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.699233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff1f1a-b659-4398-825c-6467ba780b18 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.704458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.704827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61102c8-077f-4aa0-ab45-33e22d76bac8 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.705360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.705641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.705976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] PATH_INFO: `/v3/projects/921e753c650546dc847f45600e9bc0e0/users/40e061bfb22443ae9b4191dd62ee21a1/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.708792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.709572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.709855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.710130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.721696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.723225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.723628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=921e753c650546dc847f45600e9bc0e0, user_id=40e061bfb22443ae9b4191dd62ee21a1, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.727442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:54.736921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 796/1615] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:54.746909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.750029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 820/1616] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:54 2022] PUT /identity/v3/projects/921e753c650546dc847f45600e9bc0e0/users/40e061bfb22443ae9b4191dd62ee21a1/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:54.751144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:54.758047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c58518-f250-4b89-a3e3-b268d8a354bb service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.867880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:54.875757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4f811-2c25-4169-9cd3-1bdd350a06f5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:54.877332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19800daf-9180-4550-8411-0df82515fa83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.878670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800daf-9180-4550-8411-0df82515fa83 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.879367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800daf-9180-4550-8411-0df82515fa83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.879789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19800daf-9180-4550-8411-0df82515fa83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.884926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:54.885858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:54.888584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:54.888955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:54.965282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:54.994561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:54.997958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.014981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 821/1617] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:55.016147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.027510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800daf-9180-4550-8411-0df82515fa83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.029125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19800daf-9180-4550-8411-0df82515fa83 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.032943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26a55f-5b5d-45aa-92d8-01c7e76c5441 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.041038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04628bf-253e-4599-9c9f-8068e6d9fa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.042067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04628bf-253e-4599-9c9f-8068e6d9fa1c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.042428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04628bf-253e-4599-9c9f-8068e6d9fa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.043021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04628bf-253e-4599-9c9f-8068e6d9fa1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.043729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04628bf-253e-4599-9c9f-8068e6d9fa1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.046274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04628bf-253e-4599-9c9f-8068e6d9fa1c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.048139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 797/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:55.052615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2e85-fa76-4035-8e6b-d46bd063484a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.053150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2e85-fa76-4035-8e6b-d46bd063484a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.053271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2e85-fa76-4035-8e6b-d46bd063484a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.090477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 822/1619] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:55 2022] POST /identity/v3/projects => generated 361 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:55.090477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f12d7-ab68-40b5-9f4b-2d9bd5062c72 None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.090477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f12d7-ab68-40b5-9f4b-2d9bd5062c72 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.090477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f12d7-ab68-40b5-9f4b-2d9bd5062c72 None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.092818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 823/1620] 10.222.0.17 () {54 vars in 912 bytes} [Fri Jul 15 19:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:24:55.093711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f12d7-ab68-40b5-9f4b-2d9bd5062c72 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.096964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b99b2e85-fa76-4035-8e6b-d46bd063484a None None] MFA Rules not processed for user `40e061bfb22443ae9b4191dd62ee21a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:55.097992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b99b2e85-fa76-4035-8e6b-d46bd063484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:55.189770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 798/1621] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:55.191214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.191503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.192424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.215641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:55.216233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f12d7-ab68-40b5-9f4b-2d9bd5062c72 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.217556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:55.221734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.223380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.223890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.224596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.225746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.229445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.269458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 824/1622] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:55 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:55.270588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.286855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19f7bf-914d-47e1-ba31-5250d41fc670 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.293130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.294050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.294397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.294754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.295661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.298383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.299438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 799/1623] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:24:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:55.300398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.318949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ea97d-45bb-4532-ad35-b6ce25a45459 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.324980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.325668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.325957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.326376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.327307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.328995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.350315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 800/1624] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:55.351525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.368761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 825/1625] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:24:55 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:55.368761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.393240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:55.540666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c25ae-cf5f-4bef-ad5f-67ed9039ae54 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.566257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d8d3a6-3bdd-4ce5-a611-c8297c049e5f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.579825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.584639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.588416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.588904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] PATH_INFO: `/v3/projects/6d267424616e4fb79b22ece2458c98f0/users/a7efe967e3444bbca3c0979df82a329c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.612354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d267424616e4fb79b22ece2458c98f0, user_id=a7efe967e3444bbca3c0979df82a329c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.618716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.670273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 826/1626] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:55 2022] PUT /identity/v3/projects/6d267424616e4fb79b22ece2458c98f0/users/a7efe967e3444bbca3c0979df82a329c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:55.671396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.700582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:55.735506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.736755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.737044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.740007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.861463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a0566-d052-4492-b2b6-2bb24198f106 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.878620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.879308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.879593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.879885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.880629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.884845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.953672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.953672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.956812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 827/1627] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:24:55 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:24:55.956812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.961070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f08fab-d05e-4c16-a1bd-40a11509f386 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:55.974468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:55.975460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:55.976233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:55.977625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:55.981003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:55.982869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:55.987307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 801/1628] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:55.989564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:55.998835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 828/1629] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:56.012815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:56.023733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869571d-798f-4026-b4fb-b87ae591d4e2 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.043636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.047296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.047797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.048340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.049121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb32656-e430-47ad-a243-f48430c46a2b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.051112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:56.056985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:56.065027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8461-09c0-48af-8155-46bbe0592cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.066210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8461-09c0-48af-8155-46bbe0592cfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.067258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8461-09c0-48af-8155-46bbe0592cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.069703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8461-09c0-48af-8155-46bbe0592cfb admin admin] PATH_INFO: `/v3/projects/6d267424616e4fb79b22ece2458c98f0/users/a7efe967e3444bbca3c0979df82a329c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.073653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8461-09c0-48af-8155-46bbe0592cfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d267424616e4fb79b22ece2458c98f0, user_id=a7efe967e3444bbca3c0979df82a329c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:56.080019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8461-09c0-48af-8155-46bbe0592cfb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:56.088490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 802/1630] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:56.092434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:56.109086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10747dcd-1024-4d4c-a099-3166d77e8474 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.127104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 829/1631] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:55 2022] PUT /identity/v3/projects/6d267424616e4fb79b22ece2458c98f0/users/a7efe967e3444bbca3c0979df82a329c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:56.139125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.139447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.140703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.199206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] MFA Rules not processed for user `a7efe967e3444bbca3c0979df82a329c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:56.200480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:56.214346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:56.222336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.223697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.223697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.223890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] PATH_INFO: `/v3/projects/7abc2997fa2a453299f9e69142ede002/users/08ae4497626a4fd68f76b6f4fde58388/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.242287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abc2997fa2a453299f9e69142ede002, user_id=08ae4497626a4fd68f76b6f4fde58388, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:56.245214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:56.269628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 803/1632] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:56 2022] PUT /identity/v3/projects/7abc2997fa2a453299f9e69142ede002/users/08ae4497626a4fd68f76b6f4fde58388/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:56.270552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:56.348111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efc0b0-0478-4b37-8150-8e5cc1858854 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.356768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.357632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.357903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.358190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.358939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:56.360905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:56.377227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 804/1633] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:24:56 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:24:56.378457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:56.398120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 830/1634] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:56.403177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46f53-0386-447b-8325-c985f7410a19 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.408486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af4539-0da5-452d-9725-f42b04b0e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.409505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4539-0da5-452d-9725-f42b04b0e33a admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.411344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:56.411574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4539-0da5-452d-9725-f42b04b0e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.416846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af4539-0da5-452d-9725-f42b04b0e33a admin admin] PATH_INFO: `/v3/projects/7abc2997fa2a453299f9e69142ede002/users/08ae4497626a4fd68f76b6f4fde58388/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.423790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af4539-0da5-452d-9725-f42b04b0e33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abc2997fa2a453299f9e69142ede002, user_id=08ae4497626a4fd68f76b6f4fde58388, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:56.426550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af4539-0da5-452d-9725-f42b04b0e33a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:56.467222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 805/1635] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:24:56 2022] PUT /identity/v3/projects/7abc2997fa2a453299f9e69142ede002/users/08ae4497626a4fd68f76b6f4fde58388/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:56.470339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.470630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.470914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.526456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5527616-3d05-46d9-8703-1f18fc21057c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.537110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] MFA Rules not processed for user `08ae4497626a4fd68f76b6f4fde58388`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:56.539602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:56.539735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:56.675377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.676757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.677720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.678239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:56.760229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 806/1636] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:56.839994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:56.842860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:56.865273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 831/1637] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:24:56.956305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:56.979388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695b454-1aa6-4589-b487-9dff1f4bd7b1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:56.996180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e27567-5d28-4fcd-9d11-fb77cd7fa589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:56.996865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27567-5d28-4fcd-9d11-fb77cd7fa589 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:56.997206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27567-5d28-4fcd-9d11-fb77cd7fa589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:56.997494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e27567-5d28-4fcd-9d11-fb77cd7fa589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:57.033251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e27567-5d28-4fcd-9d11-fb77cd7fa589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:57.035771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e27567-5d28-4fcd-9d11-fb77cd7fa589 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:57.079634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 807/1638] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:56 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:57.189009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:57.206768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50462772-4d7f-4734-b93f-6e389539d37f service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:57.214804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9338a1-131d-4c20-a28b-c7578298c891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:57.216590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9338a1-131d-4c20-a28b-c7578298c891 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:57.216883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9338a1-131d-4c20-a28b-c7578298c891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:57.217594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9338a1-131d-4c20-a28b-c7578298c891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:57.398720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9338a1-131d-4c20-a28b-c7578298c891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:57.406397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9338a1-131d-4c20-a28b-c7578298c891 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:57.443717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 832/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:57 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:57.792538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be459c96-50dd-4e94-9ded-8afdff97565f None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:57.793118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be459c96-50dd-4e94-9ded-8afdff97565f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:57.793118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be459c96-50dd-4e94-9ded-8afdff97565f None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:57.794955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 808/1640] 10.222.0.17 () {54 vars in 961 bytes} [Fri Jul 15 19:24:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:24:57.814710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b91e4-61f6-471d-9f56-8adc78e93e99 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:57.814710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b91e4-61f6-471d-9f56-8adc78e93e99 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:57.814710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b91e4-61f6-471d-9f56-8adc78e93e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:57.841054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2d0b91e4-61f6-471d-9f56-8adc78e93e99 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:57.842418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2d0b91e4-61f6-471d-9f56-8adc78e93e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:58.014448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:58.048741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be459c96-50dd-4e94-9ded-8afdff97565f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:58.068514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 833/1641] 10.222.0.17 () {58 vars in 1058 bytes} [Fri Jul 15 19:24:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:58.071986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d323-c41b-4aeb-bde7-b3963d98600f None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:58.072712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d323-c41b-4aeb-bde7-b3963d98600f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:58.073353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6d323-c41b-4aeb-bde7-b3963d98600f None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:58.088136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 834/1642] 10.222.0.17 () {54 vars in 912 bytes} [Fri Jul 15 19:24:58 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:24:58.090719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:58.091014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:58.091632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:58.120743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:24:58.170937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:58.174437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:58.199813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 835/1643] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:24:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:58.213768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be459c96-50dd-4e94-9ded-8afdff97565f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:58.215035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:58.229036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f473b-6c1c-4cb8-ba5a-1b5d73bfcc56 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:58.231578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:58.233077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:58.234223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:58.235081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] PATH_INFO: `/v3/users/40e061bfb22443ae9b4191dd62ee21a1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:58.236700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b33175-07d1-402e-95cf-5319e680f5a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:58.237928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b33175-07d1-402e-95cf-5319e680f5a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:58.238888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e061bfb22443ae9b4191dd62ee21a1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:58.239311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b33175-07d1-402e-95cf-5319e680f5a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:58.240038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b33175-07d1-402e-95cf-5319e680f5a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:58.242774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:58.264974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b33175-07d1-402e-95cf-5319e680f5a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:58.270060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b33175-07d1-402e-95cf-5319e680f5a1 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:58.356785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 836/1644] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:24:58.386719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e061bfb22443ae9b4191dd62ee21a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:58.405799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e061bfb22443ae9b4191dd62ee21a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:58.406511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e061bfb22443ae9b4191dd62ee21a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:58.407212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e061bfb22443ae9b4191dd62ee21a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:58.423133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e061bfb22443ae9b4191dd62ee21a1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:58.455463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 809/1645] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:24:58 2022] DELETE /identity/v3/users/40e061bfb22443ae9b4191dd62ee21a1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:24:58.460104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:58.461672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:58.461672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:58.514154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:24:58.516215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:58.630367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 837/1646] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:24:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:24:58.671151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:58.672926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:58.881852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0522efcc-1ec5-4ac8-bbe3-8e892c177d3d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:58.899819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:58.900813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:58.901422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:58.901860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:58.919184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:58.922029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:58.944156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 810/1647] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:24:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:24:59.783496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:59.784994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:24:59.803513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd83ae-f8be-4278-bc89-2438927155a0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:24:59.821269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:24:59.822292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:24:59.822654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:24:59.822995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] PATH_INFO: `/v3/projects/921e753c650546dc847f45600e9bc0e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:24:59.833120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] RBAC: Authorizing `identity:delete_project(project_id=921e753c650546dc847f45600e9bc0e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:24:59.835436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:24:59.889668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '921e753c650546dc847f45600e9bc0e0', 'request_id': 'req-94db458b-6f8b-4042-8b9d-853e1cd6e48f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:59.899517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '921e753c650546dc847f45600e9bc0e0', 'request_id': 'req-94db458b-6f8b-4042-8b9d-853e1cd6e48f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:59.908518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '921e753c650546dc847f45600e9bc0e0', 'request_id': 'req-94db458b-6f8b-4042-8b9d-853e1cd6e48f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:59.931368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] The token cache is being invalidate because project 921e753c650546dc847f45600e9bc0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:24:59.931700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:59.932457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:24:59.934619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 838/1648] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:24:59 2022] DELETE /identity/v3/projects/921e753c650546dc847f45600e9bc0e0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:24:59.938769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:24:59.956642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38ca40-d63f-408a-ae55-eef388651d4f service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:00.071682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:00.080903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:00.082009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:00.082395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:00.082986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] PATH_INFO: `/v3/users/08ae4497626a4fd68f76b6f4fde58388` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:00.086562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ae4497626a4fd68f76b6f4fde58388)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:00.094146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:00.118665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:00.120216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:00.166063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db458b-6f8b-4042-8b9d-853e1cd6e48f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:00.175084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:00.175896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:00.176452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:00.176850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] PATH_INFO: `/v3/users/e4d49f9f74ce4feda54e3eebcc2bbdf0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:00.178417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d49f9f74ce4feda54e3eebcc2bbdf0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:00.186741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: WARNING py.warnings [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:25:00.186741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: warnings.warn(msg) Jul 15 19:25:00.186741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: Jul 15 19:25:00.203281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:00.235988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ae4497626a4fd68f76b6f4fde58388'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.261114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ae4497626a4fd68f76b6f4fde58388'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.261790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ae4497626a4fd68f76b6f4fde58388'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.264985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ae4497626a4fd68f76b6f4fde58388'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.288361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-21d360ec-df79-44c4-b299-c5baf1e9b473 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ae4497626a4fd68f76b6f4fde58388'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.403062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 811/1649] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:24:59 2022] DELETE /identity/v3/users/08ae4497626a4fd68f76b6f4fde58388 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:00.466883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d49f9f74ce4feda54e3eebcc2bbdf0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.477764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:00.495428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d49f9f74ce4feda54e3eebcc2bbdf0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.496845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d49f9f74ce4feda54e3eebcc2bbdf0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.497967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d49f9f74ce4feda54e3eebcc2bbdf0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.564490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d49f9f74ce4feda54e3eebcc2bbdf0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:00.627195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 839/1650] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:00 2022] DELETE /identity/v3/users/e4d49f9f74ce4feda54e3eebcc2bbdf0 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:00.631582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:00.631881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:00.632718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:00.677256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:00.678414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:00.813342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 812/1651] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:00.874730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:00.875977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:01.052465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd09c76-9e45-4bdc-899f-0cc1fe8fd3b1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:01.069497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a075441-88b4-455a-b830-b7282c81223c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:01.071186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075441-88b4-455a-b830-b7282c81223c service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:01.071621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075441-88b4-455a-b830-b7282c81223c service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:01.071907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a075441-88b4-455a-b830-b7282c81223c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:01.092034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075441-88b4-455a-b830-b7282c81223c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:01.093544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a075441-88b4-455a-b830-b7282c81223c service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:01.120946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 840/1652] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:01.256501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:01.261039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:01.283117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc775a-e53a-46fc-a6ce-fdb7a56715d2 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:01.290439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:01.291394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:01.291860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:01.292225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] PATH_INFO: `/v3/users/14ba4ed80c014715a763d9b542efe534` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:01.293574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14ba4ed80c014715a763d9b542efe534)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:01.296938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:01.383246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14ba4ed80c014715a763d9b542efe534'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:01.423017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14ba4ed80c014715a763d9b542efe534'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:01.428624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14ba4ed80c014715a763d9b542efe534'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:01.431062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14ba4ed80c014715a763d9b542efe534'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:01.452717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14ba4ed80c014715a763d9b542efe534'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:01.464653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:01.484767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 813/1653] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:01 2022] DELETE /identity/v3/users/14ba4ed80c014715a763d9b542efe534 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:01.490819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:01.490819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:01.490819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:01.534873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:01.539001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:01.675371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 841/1654] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:01.727351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:01.732408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:01.880406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7881e23-29b4-40ed-bf95-2ccccae8d5e8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:01.885903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e4d60a-0612-4fbf-ad7d-b10d58ae89e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:01.886597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4d60a-0612-4fbf-ad7d-b10d58ae89e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:01.886881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4d60a-0612-4fbf-ad7d-b10d58ae89e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:01.887212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4d60a-0612-4fbf-ad7d-b10d58ae89e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:01.916735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4d60a-0612-4fbf-ad7d-b10d58ae89e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:01.918583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e4d60a-0612-4fbf-ad7d-b10d58ae89e4 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:01.945442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 814/1655] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:03.172763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:03.176727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:03.206553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f34ce-96f3-4c3a-b561-9bd9cd08be1d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:03.227070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:03.229908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:03.232152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:03.232540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] PATH_INFO: `/v3/projects/7abc2997fa2a453299f9e69142ede002` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:03.236239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7abc2997fa2a453299f9e69142ede002)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:03.240703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:03.390571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7abc2997fa2a453299f9e69142ede002', 'request_id': 'req-c427f148-69ab-46cd-bf91-5620cc4bcb1a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:03.402644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7abc2997fa2a453299f9e69142ede002', 'request_id': 'req-c427f148-69ab-46cd-bf91-5620cc4bcb1a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:03.413095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7abc2997fa2a453299f9e69142ede002', 'request_id': 'req-c427f148-69ab-46cd-bf91-5620cc4bcb1a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:03.421077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:03.429875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] The token cache is being invalidate because project 7abc2997fa2a453299f9e69142ede002 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:03.431617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:03.433205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c427f148-69ab-46cd-bf91-5620cc4bcb1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:03.436061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 842/1656] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:03 2022] DELETE /identity/v3/projects/7abc2997fa2a453299f9e69142ede002 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:03.442359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:03.443075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:03.443075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:03.480699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:03.483825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:03.656882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 815/1657] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:03.665455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:03.666202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:03.666202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:03.708102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:03.709353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:03.722288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 843/1658] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:03.726876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:03.742631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8746054a-6aa8-45e6-9148-d8938147d337 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:03.747987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:03.749123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:03.749511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:03.749966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:03.750827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:03.751466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:03.753630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:03.772181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 816/1659] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:25:03.773489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:03.789858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352d759-31ba-4469-b80d-a72dd3de0a4a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:03.802241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:03.802941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:03.803238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:03.803544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:03.804784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:03.806796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:03.883362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 844/1660] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:03 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:03.884883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:03.911615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721f0b7-6984-4bfd-80f1-5e8ddf56c7ef admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.042630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.046356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.047097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.047771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.050992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.058974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.131495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 817/1661] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:03 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:04.134390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:04.187069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc14b9-f890-45b5-be40-47a02d8a98d7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.206768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.208223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.209692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.212608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.214266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.226391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:04.230728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:04.243152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.287106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae53c51-1ff7-45c8-b000-ecfbaf9371dc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.308012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 845/1662] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:04.313441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:04.324669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.325864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.328567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.329081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] PATH_INFO: `/v3/projects/cb15403b3b5d462ab74cee11b5521792` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.330671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb15403b3b5d462ab74cee11b5521792)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.334507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: WARNING py.warnings [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 15 19:25:04.334507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: warnings.warn(msg) Jul 15 19:25:04.334507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: Jul 15 19:25:04.340689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.348349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d7446-b332-4f00-aaa2-390c8e71c7fc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.349422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:04.374884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.375332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.375794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.376295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.377204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.379027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.412014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 846/1663] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:04.416297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:04.442857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0fdc2-a818-40bc-8a9b-eb494c3d9e82 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.464409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6476a37-430a-45dd-befa-7fda3501074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.465328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476a37-430a-45dd-befa-7fda3501074b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.465620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476a37-430a-45dd-befa-7fda3501074b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.465897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6476a37-430a-45dd-befa-7fda3501074b admin admin] PATH_INFO: `/v3/projects/ba7674f69a564c518885b1e309d9906e/users/cf5847bc4c7647a9b44d674975ff44d3/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.508054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6476a37-430a-45dd-befa-7fda3501074b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba7674f69a564c518885b1e309d9906e, user_id=cf5847bc4c7647a9b44d674975ff44d3, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.511187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6476a37-430a-45dd-befa-7fda3501074b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.532524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb15403b3b5d462ab74cee11b5521792', 'request_id': 'req-f4e48169-d897-4040-9173-50aa96b658e8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:04.560177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 847/1664] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:04 2022] PUT /identity/v3/projects/ba7674f69a564c518885b1e309d9906e/users/cf5847bc4c7647a9b44d674975ff44d3/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:04.583417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.583417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.583943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.602940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb15403b3b5d462ab74cee11b5521792', 'request_id': 'req-f4e48169-d897-4040-9173-50aa96b658e8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:04.622405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb15403b3b5d462ab74cee11b5521792', 'request_id': 'req-f4e48169-d897-4040-9173-50aa96b658e8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:04.657602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] MFA Rules not processed for user `cf5847bc4c7647a9b44d674975ff44d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:04.661134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:04.666570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] The token cache is being invalidate because project cb15403b3b5d462ab74cee11b5521792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:04.672697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:04.683532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f4e48169-d897-4040-9173-50aa96b658e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:04.690209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 818/1665] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:04 2022] DELETE /identity/v3/projects/cb15403b3b5d462ab74cee11b5521792 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:04.691926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdff62-0241-4785-b557-066d181268e6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.692439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdff62-0241-4785-b557-066d181268e6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.692887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdff62-0241-4785-b557-066d181268e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.717018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-34cdff62-0241-4785-b557-066d181268e6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:04.718549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34cdff62-0241-4785-b557-066d181268e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:04.855859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 848/1666] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:04.859494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:04.864002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:04.889530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 819/1667] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:04.890731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdff62-0241-4785-b557-066d181268e6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:04.893097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34cdff62-0241-4785-b557-066d181268e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:04.901452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21236e57-d2f7-4c44-96cc-728b48ccb11f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.906591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdff62-0241-4785-b557-066d181268e6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:04.914915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.916397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.916826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.917275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] PATH_INFO: `/v3/projects/4069de5b46764795816dcf7d6f525dcf` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.918144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d71c7d-f0f6-45e8-ac8d-d0fd5a526974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:04.918825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71c7d-f0f6-45e8-ac8d-d0fd5a526974 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.919076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4069de5b46764795816dcf7d6f525dcf)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.919266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71c7d-f0f6-45e8-ac8d-d0fd5a526974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.919372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d71c7d-f0f6-45e8-ac8d-d0fd5a526974 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:04.920057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71c7d-f0f6-45e8-ac8d-d0fd5a526974 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:04.921921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.922364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d71c7d-f0f6-45e8-ac8d-d0fd5a526974 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:04.988239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 849/1668] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:04 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:04.989600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:04.989925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:04.992310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.028075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:05.029787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:05.037067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4069de5b46764795816dcf7d6f525dcf', 'request_id': 'req-f58540f8-a7b1-4313-addc-697502d4c1c8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.064658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4069de5b46764795816dcf7d6f525dcf', 'request_id': 'req-f58540f8-a7b1-4313-addc-697502d4c1c8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.090917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4069de5b46764795816dcf7d6f525dcf', 'request_id': 'req-f58540f8-a7b1-4313-addc-697502d4c1c8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.097061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:05.125222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] The token cache is being invalidate because project 4069de5b46764795816dcf7d6f525dcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:05.125644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 850/1669] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:05.125754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.127264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.129882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.131513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 820/1670] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:04 2022] DELETE /identity/v3/projects/4069de5b46764795816dcf7d6f525dcf => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:05.135228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.141018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:05.144312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8406046a-9698-411b-bbc4-594a2d97db86 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.151020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.151908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.152226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.152510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.153214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.155904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.190345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 851/1671] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:05 2022] POST /identity/v3/users => generated 291 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:05.191974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58540f8-a7b1-4313-addc-697502d4c1c8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.194175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.196702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:05.204356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.205061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.205350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.205637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.206387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:05.206699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.208376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.235519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:05.237228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7ba7f-3600-4917-ba3d-5b89e5dcff62 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.251925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.252954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.253395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.253682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] PATH_INFO: `/v3/users/a7efe967e3444bbca3c0979df82a329c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.257216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7efe967e3444bbca3c0979df82a329c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.263862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.266749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 821/1672] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:25:05.269070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.277080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:05.317216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3e3b2-33f2-4dea-8a00-6971199bc2a2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.331974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.332707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.332982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.333256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.333998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.336615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.363727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 822/1673] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:05.366111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.383552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c1133-2586-48e9-b6ba-3575a0a82f20 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.390437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.391243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.391607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.392005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.393304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.395779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.397964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7efe967e3444bbca3c0979df82a329c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.418710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7efe967e3444bbca3c0979df82a329c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.419580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7efe967e3444bbca3c0979df82a329c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.420767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7efe967e3444bbca3c0979df82a329c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.450823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 823/1674] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:05 2022] POST /identity/v3/projects => generated 363 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:05.451811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.453431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7efe967e3444bbca3c0979df82a329c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:05.454615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:05.482878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 852/1675] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:05 2022] DELETE /identity/v3/users/a7efe967e3444bbca3c0979df82a329c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:05.489003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.492461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:05.626818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe5f96-5509-4d5f-ab62-3ebf5fc5be8d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.628757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebb4a11-3c5f-4779-a61e-5296135c1c81 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.635076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.636003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.637196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.637549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.638340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.639746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.640694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.640974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.641079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.641248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] PATH_INFO: `/v3/projects/7f1f2dfc643849329be73437304dc7d9/users/70c8eb104597432590391015da39068d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.678004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1f2dfc643849329be73437304dc7d9, user_id=70c8eb104597432590391015da39068d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.682844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 853/1676] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:05 2022] POST /identity/v3/users => generated 282 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:05.685159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.689141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.702549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d48dbe-0028-4048-bd04-76a7ea18356c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.721612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.722324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.722616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.722897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.723638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.725315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.738433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 824/1677] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:05 2022] PUT /identity/v3/projects/7f1f2dfc643849329be73437304dc7d9/users/70c8eb104597432590391015da39068d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:05.746790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.753522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 854/1678] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:05.754477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.780127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bf62ff-6b3c-48b7-b276-f0d296a56a17 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.822805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:05.900508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.901200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.901487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.901764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.902505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.907025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.928528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 855/1679] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:05.933568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.934206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae199136-0983-4ece-893b-194201d59d16 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.947788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.948063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.948349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.948628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.949346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:05.950981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:05.962730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57be525-0499-462b-95bd-b5de84611500 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:05.972945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 825/1680] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:05 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:05.977480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:05.978412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301deeca-702e-489f-9f87-7397b6f0f945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:05.979627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301deeca-702e-489f-9f87-7397b6f0f945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:05.979799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301deeca-702e-489f-9f87-7397b6f0f945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:05.980036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301deeca-702e-489f-9f87-7397b6f0f945 admin admin] PATH_INFO: `/v3/projects/c8a89ff742e649dc8e78c28913ff947f/users/4852d00378a04420a74aea58b8e5f07c/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:05.997673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3711de61-1585-473c-9a1e-20f9fe7ec2c7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.013052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957556-2157-4e26-b21b-4ef09e33912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.013865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957556-2157-4e26-b21b-4ef09e33912d admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.014151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957556-2157-4e26-b21b-4ef09e33912d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.014426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957556-2157-4e26-b21b-4ef09e33912d admin admin] PATH_INFO: `/v3/projects/7f1f2dfc643849329be73437304dc7d9/users/70c8eb104597432590391015da39068d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.018278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6957556-2157-4e26-b21b-4ef09e33912d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1f2dfc643849329be73437304dc7d9, user_id=70c8eb104597432590391015da39068d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.023570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6957556-2157-4e26-b21b-4ef09e33912d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.027175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301deeca-702e-489f-9f87-7397b6f0f945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8a89ff742e649dc8e78c28913ff947f, user_id=4852d00378a04420a74aea58b8e5f07c, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.030317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301deeca-702e-489f-9f87-7397b6f0f945 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.050913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 826/1681] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:05 2022] PUT /identity/v3/projects/7f1f2dfc643849329be73437304dc7d9/users/70c8eb104597432590391015da39068d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:06.061636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.061961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.062315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.077975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 856/1682] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:05 2022] PUT /identity/v3/projects/c8a89ff742e649dc8e78c28913ff947f/users/4852d00378a04420a74aea58b8e5f07c/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:06.094008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.094401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.094630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.112989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] MFA Rules not processed for user `70c8eb104597432590391015da39068d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:06.114236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:06.174091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:06.190122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] MFA Rules not processed for user `4852d00378a04420a74aea58b8e5f07c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:06.205572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:06.318744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 827/1683] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:06 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:06.326013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:06.327729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:06.387650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 857/1684] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:06.392757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:06.456762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87673490-1a7d-4e22-8acc-13a0bd2fc699 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.466625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.468462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.469675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.469675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.494699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.498225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.523618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 828/1685] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:06 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:06.536240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916d0dd1-406c-4010-9c0c-d25af57e4d97 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.543312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.544779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.544888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.545265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.546048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.548177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.606779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 858/1686] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:06 2022] POST /identity/v3/projects => generated 363 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:06.608371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:06.608483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:06.610139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:06.626551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a422d-761d-447b-ac5a-08cc143271f1 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.637307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:06.698172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.700855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.701976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.702468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.705091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:06.760415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5bd55-c67c-4761-9d80-196f0078f888 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.775666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.776156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.776767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.777108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.777660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.789417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.829092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.829092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.829937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 859/1687] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:06 2022] POST /identity/v3/users => generated 288 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:06.832324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:06.850661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 829/1688] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:06.852701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e681c6d-8e59-413f-b2b2-690d4f673a78 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.860781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.862303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.862699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.863073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.864820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.868003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:06.892595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:06.896888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 860/1689] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:06.917976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418bacb-4aff-4deb-8d2b-e4c154a2470f service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:06.927815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66db5ca-34e4-4332-ac8e-794af841084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:06.929755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db5ca-34e4-4332-ac8e-794af841084d admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:06.930025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db5ca-34e4-4332-ac8e-794af841084d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:06.930305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66db5ca-34e4-4332-ac8e-794af841084d admin admin] PATH_INFO: `/v3/projects/86bdf5ba521d4ac394d27005fe9f2860/users/de8a545f67d14bf1a2a53b085a04fafd/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:06.949987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:06.957973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66db5ca-34e4-4332-ac8e-794af841084d admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bdf5ba521d4ac394d27005fe9f2860, user_id=de8a545f67d14bf1a2a53b085a04fafd, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:06.966973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66db5ca-34e4-4332-ac8e-794af841084d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:07.004164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 830/1690] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:06 2022] PUT /identity/v3/projects/86bdf5ba521d4ac394d27005fe9f2860/users/de8a545f67d14bf1a2a53b085a04fafd/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:07.010101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:07.025683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbb9aed-1dad-4a5e-a68f-7dd45a4f7fe6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:07.103805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:07.104734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:07.105104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:07.139242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] MFA Rules not processed for user `837f13aef58b455e95372c6f106d5807`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:07.141078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:07.162623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:07.165025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:07.165777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:07.166969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:07.169937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:07.189459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:07.218324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 861/1691] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:07.219507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:07.245219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361e2a6-7974-4d8c-990b-d9faa73350bf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:07.257360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca949b-72b8-4c1d-81a7-91bced0ac7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:07.258722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca949b-72b8-4c1d-81a7-91bced0ac7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:07.266720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca949b-72b8-4c1d-81a7-91bced0ac7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:07.266720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca949b-72b8-4c1d-81a7-91bced0ac7cc admin admin] PATH_INFO: `/v3/projects/86bdf5ba521d4ac394d27005fe9f2860/users/de8a545f67d14bf1a2a53b085a04fafd/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:07.266720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca949b-72b8-4c1d-81a7-91bced0ac7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bdf5ba521d4ac394d27005fe9f2860, user_id=de8a545f67d14bf1a2a53b085a04fafd, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:07.268120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca949b-72b8-4c1d-81a7-91bced0ac7cc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:07.269335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 831/1692] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:07.276491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:07.287166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:07.332786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 862/1693] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:07 2022] PUT /identity/v3/projects/86bdf5ba521d4ac394d27005fe9f2860/users/de8a545f67d14bf1a2a53b085a04fafd/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:07.345800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:07.346090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:07.346374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:07.417624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] MFA Rules not processed for user `de8a545f67d14bf1a2a53b085a04fafd`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:07.419528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:07.467142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e30f96-2987-4463-bfc3-964de1932c24 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:07.627498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:07.661234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:07.661926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:07.662214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:07.662495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:07.668735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 863/1694] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:07 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:07.670317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:07.674016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:07.693876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:07.792843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:07.794495 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:07.820791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 832/1695] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:07.821901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:07.869126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010d48a-4ae3-425a-b1f2-90d028668780 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:07.880293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:07.884162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:07.884739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:07.885164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] PATH_INFO: `/v3/projects/6d267424616e4fb79b22ece2458c98f0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:07.886891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d267424616e4fb79b22ece2458c98f0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:07.889274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:07.985135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d267424616e4fb79b22ece2458c98f0', 'request_id': 'req-4fa424b3-9791-4504-835d-f819375877a0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:08.001139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc177920-b2a0-42d4-93e0-1706e714d074 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:08.015320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d267424616e4fb79b22ece2458c98f0', 'request_id': 'req-4fa424b3-9791-4504-835d-f819375877a0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:08.023514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d267424616e4fb79b22ece2458c98f0', 'request_id': 'req-4fa424b3-9791-4504-835d-f819375877a0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:08.045506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] The token cache is being invalidate because project 6d267424616e4fb79b22ece2458c98f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:08.045801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:08.046781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4fa424b3-9791-4504-835d-f819375877a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:08.052510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 864/1696] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:07 2022] DELETE /identity/v3/projects/6d267424616e4fb79b22ece2458c98f0 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:08.060852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42536241-4926-47e8-acd6-5d244917aa8b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.061352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42536241-4926-47e8-acd6-5d244917aa8b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.061798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42536241-4926-47e8-acd6-5d244917aa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.082083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-42536241-4926-47e8-acd6-5d244917aa8b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:08.084045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-42536241-4926-47e8-acd6-5d244917aa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:08.120425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:08.122998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.124410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.124758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.130841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:08.219666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 865/1697] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:08.222635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.223050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.223385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.267301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:08.268768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:08.290617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 866/1698] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:08.291793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:08.294068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:08.294292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:08.315906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d15b23-d705-47e2-940a-724ac4c85c8e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:08.322596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 833/1699] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:07 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:08.327880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:08.329326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.329849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.330373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.331485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:08.332222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:08.334401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:08.391041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:08.391213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 867/1700] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:25:08.405633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd216861-7b86-4c2e-9562-eb45e0408495 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:08.418688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:08.419500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.419848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.420170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.422813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:08.439006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:08.502832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:08.511773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 834/1701] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:08 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:08.518343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:08.543637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f91972-3112-497d-a564-a1301e088bec admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:08.559827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:08.757790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:08.758601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.758891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.759265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.759986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:08.762130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:08.851088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 868/1702] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:08 2022] POST /identity/v3/users => generated 294 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:08.852013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:08.867511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40075b-fc53-4077-b720-d56458d660d1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:08.882177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:08.885349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.885641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.885932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.890617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:08.895942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:08.937352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 835/1703] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:08.938460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:08.954445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b710adc-d28c-4f77-9c58-543665c02bdc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:08.971074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:08.972457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:08.972854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:08.973254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:08.974110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:08.976871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:09.020336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:09.020974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 869/1704] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:09.042093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00a37a-cbd6-4493-8a39-9c118b2cb9aa admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:09.055962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92d8f6-a8fc-4a32-bb13-5a2f60e5736a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:09.061802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92d8f6-a8fc-4a32-bb13-5a2f60e5736a admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:09.061802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92d8f6-a8fc-4a32-bb13-5a2f60e5736a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:09.062007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92d8f6-a8fc-4a32-bb13-5a2f60e5736a admin admin] PATH_INFO: `/v3/projects/3934c717927341e785dd540b2131c6fb/users/06ac460638f149b19a74e972f7e44c29/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:09.101342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92d8f6-a8fc-4a32-bb13-5a2f60e5736a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3934c717927341e785dd540b2131c6fb, user_id=06ac460638f149b19a74e972f7e44c29, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:09.108114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92d8f6-a8fc-4a32-bb13-5a2f60e5736a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:09.115002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:09.118064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:09.142851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 836/1705] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:09 2022] PUT /identity/v3/projects/3934c717927341e785dd540b2131c6fb/users/06ac460638f149b19a74e972f7e44c29/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:09.158970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:09.159270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:09.159595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:09.168341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d32bc51-9ee2-4e9a-8c63-8f6b8fc0d19e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:09.223360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] MFA Rules not processed for user `06ac460638f149b19a74e972f7e44c29`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:09.224761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:09.348202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:09.349712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:09.349998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:09.350281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] PATH_INFO: `/v3/users/4852d00378a04420a74aea58b8e5f07c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:09.355539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4852d00378a04420a74aea58b8e5f07c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:09.357586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:09.378144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 837/1706] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:09.383786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:09.398021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5adef0c-aab9-4f91-8852-e88274d65307 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:09.403363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:09.404055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:09.404384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:09.404666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:09.405567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:09.407721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:09.472449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 838/1707] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:09 2022] POST /identity/v3/projects => generated 387 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:09.483581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:09.511348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4852d00378a04420a74aea58b8e5f07c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:09.543434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4852d00378a04420a74aea58b8e5f07c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:09.544592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4852d00378a04420a74aea58b8e5f07c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:09.546560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4852d00378a04420a74aea58b8e5f07c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:09.594780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-07618e1c-a0c9-4396-8f29-6f97a1d327fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4852d00378a04420a74aea58b8e5f07c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:09.644671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 870/1708] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:09 2022] DELETE /identity/v3/users/4852d00378a04420a74aea58b8e5f07c => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:09.656014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:09.656124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:09.656381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:09.687529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:09.734675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:09.735927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:09.756466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5580e49-b6d6-42e1-b93d-69e04291dd2f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:09.894071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:09.965824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e590a2-270e-4394-87cc-fe2b39dcfc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:09.966869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e590a2-270e-4394-87cc-fe2b39dcfc23 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:09.967475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e590a2-270e-4394-87cc-fe2b39dcfc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:09.968041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e590a2-270e-4394-87cc-fe2b39dcfc23 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:09.969092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e590a2-270e-4394-87cc-fe2b39dcfc23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:09.971350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e590a2-270e-4394-87cc-fe2b39dcfc23 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:09.983292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 871/1709] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:10.015006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 839/1710] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:09 2022] POST /identity/v3/users => generated 294 bytes in 542 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:10.019127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:10.020729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:10.048118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.048422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.048703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.057124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b346020-108f-4354-bcc5-e169827fa898 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:10.077294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:10.078307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.078810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.079717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.079984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:10.081737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:10.105627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 872/1711] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:10.106690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:10.110232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:10.114261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] MFA Rules not processed for user `01120eedaa844f24a66024df8d3b2271`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:10.115571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:10.359052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640734-8351-4ea3-8ea7-739c3216f258 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:10.371540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:10.372268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.372558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.372842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.389404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 840/1712] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:10.390488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:10.410135 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:10.411628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:10.427965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b846a-416c-41b6-8cd0-9a48ce2ae1a5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:10.440907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:10.441784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.442041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.442137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 873/1713] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:10.442961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:10.445868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.446120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:10.446703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:10.448395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:10.495991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 841/1714] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:10.497234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:10.522845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913db3c2-9eee-46e3-964e-2b294a01d80e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:10.535350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dfc17-603b-472b-9afe-7d5becb1233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:10.536301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dfc17-603b-472b-9afe-7d5becb1233e admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.536592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dfc17-603b-472b-9afe-7d5becb1233e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.537084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dfc17-603b-472b-9afe-7d5becb1233e admin admin] PATH_INFO: `/v3/projects/57000d0f3e5b478ba26bb77a15d24c00/users/81680a6f5b8d476daa4781d73c2f6908/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.576003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dfc17-603b-472b-9afe-7d5becb1233e admin admin] RBAC: Authorizing `identity:create_grant(project_id=57000d0f3e5b478ba26bb77a15d24c00, user_id=81680a6f5b8d476daa4781d73c2f6908, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:10.578926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dfc17-603b-472b-9afe-7d5becb1233e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:10.621113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 842/1715] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:10 2022] PUT /identity/v3/projects/57000d0f3e5b478ba26bb77a15d24c00/users/81680a6f5b8d476daa4781d73c2f6908/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:10.622863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.623157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.623701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.675356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] MFA Rules not processed for user `81680a6f5b8d476daa4781d73c2f6908`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:10.676908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:10.677790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b94667-f3db-4c68-846a-a440e25c1d3e service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:10.746895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:10.809918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 843/1716] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:10.814779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:10.818465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:10.820521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:10.820958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:10.821380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:10.849252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:10.956051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:10.957740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:10.983447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 874/1717] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:11.008896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25401543-44bc-4668-9197-c3a0bae3dd87 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:11.022609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:11.023317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.023599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.023879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.024624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:11.026744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:11.100140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 844/1718] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:10 2022] POST /identity/v3/projects => generated 387 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:11.104724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:11.125801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16d4343-84ce-4ca4-a15b-34ab742fd552 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:11.290473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:11.291159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.291451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.291729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.292466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:11.294438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:11.349361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 875/1719] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:11 2022] POST /identity/v3/users => generated 300 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:11.350877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:11.379039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e31d76-5c13-469b-aca7-8170079b134b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:11.393333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:11.395279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.396175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.397184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.398986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:11.408759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:11.432719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 845/1720] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:11.439487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:11.454314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e2c61-4d80-4e54-aa9c-46d7132f7046 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:11.459400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:11.460574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.461037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.461511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] PATH_INFO: `/v3/projects/8c931d0876e8497ab0850a15c2d3e030/users/a71f59b584104ff1bb0027d80e999592/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.491898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c931d0876e8497ab0850a15c2d3e030, user_id=a71f59b584104ff1bb0027d80e999592, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:11.494984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:11.521960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 876/1721] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:11 2022] PUT /identity/v3/projects/8c931d0876e8497ab0850a15c2d3e030/users/a71f59b584104ff1bb0027d80e999592/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:11.523513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:11.548868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e30e08d-c9a5-47ed-a536-3b744d7d4ceb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:11.635528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687361e2-bcd9-498c-8709-e50e9ae32818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:11.636572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687361e2-bcd9-498c-8709-e50e9ae32818 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.636941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687361e2-bcd9-498c-8709-e50e9ae32818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.637288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687361e2-bcd9-498c-8709-e50e9ae32818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.638293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687361e2-bcd9-498c-8709-e50e9ae32818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:11.642820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687361e2-bcd9-498c-8709-e50e9ae32818 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:11.674449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 846/1722] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:11.678952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:11.704802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618b3fc3-0106-4aa9-87c7-0f5af5429e27 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:11.710468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:11.711433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.711717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.711995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] PATH_INFO: `/v3/projects/8c931d0876e8497ab0850a15c2d3e030/users/a71f59b584104ff1bb0027d80e999592/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.724978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c931d0876e8497ab0850a15c2d3e030, user_id=a71f59b584104ff1bb0027d80e999592, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:11.733813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:11.790036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 877/1723] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:11 2022] PUT /identity/v3/projects/8c931d0876e8497ab0850a15c2d3e030/users/a71f59b584104ff1bb0027d80e999592/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:11.793289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:11.793766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:11.794263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:11.867146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] MFA Rules not processed for user `a71f59b584104ff1bb0027d80e999592`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:11.868003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:12.022421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 847/1724] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:12.035276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:12.037871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:12.107819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:12.193960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123e47a-8227-4bac-bce9-d2c74f2f391c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:12.215216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9205ff5d-160f-4600-9317-2a640cffb9be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:12.215216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205ff5d-160f-4600-9317-2a640cffb9be service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:12.215216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205ff5d-160f-4600-9317-2a640cffb9be service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:12.215216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9205ff5d-160f-4600-9317-2a640cffb9be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:12.253092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205ff5d-160f-4600-9317-2a640cffb9be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:12.261390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9205ff5d-160f-4600-9317-2a640cffb9be service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:12.302236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 878/1725] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:12.763299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:12.766641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:12.823039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:12.964937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd16b54-3ab9-46d3-aaa4-508c4d36b018 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:12.983963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:12.985259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:12.985757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:12.986332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] PATH_INFO: `/v3/users/cf5847bc4c7647a9b44d674975ff44d3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:12.988060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5847bc4c7647a9b44d674975ff44d3)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:12.992036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:13.148246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5847bc4c7647a9b44d674975ff44d3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.176442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5847bc4c7647a9b44d674975ff44d3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.177071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5847bc4c7647a9b44d674975ff44d3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.177770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5847bc4c7647a9b44d674975ff44d3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.193463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5847bc4c7647a9b44d674975ff44d3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.238403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 848/1726] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:12 2022] DELETE /identity/v3/users/cf5847bc4c7647a9b44d674975ff44d3 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:13.244631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:13.245161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:13.245161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:13.255298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:13.257859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:13.298165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:13.299507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:13.476512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4b118-ed95-44df-a640-cf14245e2e3e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:13.493013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:13.493920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:13.494203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:13.494481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] PATH_INFO: `/v3/projects/c8a89ff742e649dc8e78c28913ff947f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:13.496810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8a89ff742e649dc8e78c28913ff947f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:13.499186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:13.561325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 879/1727] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:13.577842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:13.618435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8a89ff742e649dc8e78c28913ff947f', 'request_id': 'req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.634928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:13.636654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:13.644721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8a89ff742e649dc8e78c28913ff947f', 'request_id': 'req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.669885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8a89ff742e649dc8e78c28913ff947f', 'request_id': 'req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.711665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] The token cache is being invalidate because project c8a89ff742e649dc8e78c28913ff947f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:13.711956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.712708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:13.714332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 849/1728] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:13 2022] DELETE /identity/v3/projects/c8a89ff742e649dc8e78c28913ff947f => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:13.715147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:13.720012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:13.821628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6567f3-c7df-4fd6-a501-86d7a9607a04 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:13.907655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcb230-013b-4ecd-91be-f3bf3ffcf291 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:13.917192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:13.917874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:13.918158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:13.918624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:13.927612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:13.995458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:13.996346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:13.996638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:13.996913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:14.002397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:14.142416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:14.145057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:14.219180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 850/1729] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:14.224376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:14.234543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:14.235744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:14.297415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71694d84-ef80-4afd-baa8-443411dcad03 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:14.335448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:14.337028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:14.337427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:14.337888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] PATH_INFO: `/v3/users/de8a545f67d14bf1a2a53b085a04fafd` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:14.350000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de8a545f67d14bf1a2a53b085a04fafd)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:14.361729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:14.377287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:14.377969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b30bfa-4229-4c65-a5a5-bc0b7ce90021 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:14.461262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 880/1730] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 827 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:14.689632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de8a545f67d14bf1a2a53b085a04fafd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:14.748868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:14.749310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:14.750327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:14.759820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de8a545f67d14bf1a2a53b085a04fafd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:14.762438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de8a545f67d14bf1a2a53b085a04fafd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:14.765656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de8a545f67d14bf1a2a53b085a04fafd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:14.825729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-26941768-b282-46d6-961b-c0ab3f1694f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de8a545f67d14bf1a2a53b085a04fafd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:14.838220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:14.893325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] MFA Rules not processed for user `d06d1b5c0eff4645bf7fb43f8921bb17`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:14.899609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:14.937084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 851/1731] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:14 2022] DELETE /identity/v3/users/de8a545f67d14bf1a2a53b085a04fafd => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:15.097524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 881/1732] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:15.102274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:15.102774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:15.103191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:15.145746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] MFA Rules not processed for user `d06d1b5c0eff4645bf7fb43f8921bb17`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:15.147741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:15.367801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 852/1733] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:15.379907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:15.382509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:15.615430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14d2d7-763e-4c97-ab63-455764e08769 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:15.772686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:15.791490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:15.792328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:15.792617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:15.793147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:15.813356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:15.817526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:15.860593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 882/1734] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:16.711363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:16.713567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:16.949292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39365f53-cac0-4036-9a21-20a54c09155a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:16.957782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:16.958964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:16.959421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:16.960710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] PATH_INFO: `/v3/projects/ba7674f69a564c518885b1e309d9906e` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:16.962191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba7674f69a564c518885b1e309d9906e)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:16.965547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:17.065371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba7674f69a564c518885b1e309d9906e', 'request_id': 'req-ada773bd-6ab6-4e7b-8468-18a493604d78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.074603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba7674f69a564c518885b1e309d9906e', 'request_id': 'req-ada773bd-6ab6-4e7b-8468-18a493604d78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.106711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba7674f69a564c518885b1e309d9906e', 'request_id': 'req-ada773bd-6ab6-4e7b-8468-18a493604d78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.125236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] The token cache is being invalidate because project ba7674f69a564c518885b1e309d9906e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:17.125739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.126879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ada773bd-6ab6-4e7b-8468-18a493604d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.130747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 853/1735] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:16 2022] DELETE /identity/v3/projects/ba7674f69a564c518885b1e309d9906e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:17.134999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:17.154302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74661a01-7ca0-4bb2-9446-f37d624f191e service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:17.318184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131543-54af-406f-baff-278d4ccba99b None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:17.318519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131543-54af-406f-baff-278d4ccba99b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:17.318998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf131543-54af-406f-baff-278d4ccba99b None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:17.325946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 854/1736] 10.222.0.17 () {54 vars in 912 bytes} [Fri Jul 15 19:25:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:25:17.333569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527e954-0aae-47fe-85f7-080ad90fb69e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:17.333923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527e954-0aae-47fe-85f7-080ad90fb69e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:17.335357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527e954-0aae-47fe-85f7-080ad90fb69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:17.359711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:17.361269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:17.361631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:17.361986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] PATH_INFO: `/v3/users/70c8eb104597432590391015da39068d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:17.368255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c8eb104597432590391015da39068d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:17.376720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:17.442476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:17.467805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e527e954-0aae-47fe-85f7-080ad90fb69e None None] MFA Rules not processed for user `9f135ce75fbc4b12977cc8f009083ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:17.472841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e527e954-0aae-47fe-85f7-080ad90fb69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:17.562687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c8eb104597432590391015da39068d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.574024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c8eb104597432590391015da39068d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.574920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c8eb104597432590391015da39068d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.575647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c8eb104597432590391015da39068d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.610513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13eaf9f0-a5cc-427d-abed-3aa139a3d031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c8eb104597432590391015da39068d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:17.633112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 855/1737] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:25:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:17.674996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:17.675301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:17.675458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:17.684791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 883/1738] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:17 2022] DELETE /identity/v3/users/70c8eb104597432590391015da39068d => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:17.718038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] MFA Rules not processed for user `025ea412ccfa455aa84d051244902cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:17.719279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:17.817747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 856/1739] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:17.830208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:17.833536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:17.834051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:17.893614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:17.898086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:17.910540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] MFA Rules not processed for user `025ea412ccfa455aa84d051244902cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:17.916063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:17.941829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 884/1740] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:17.964449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:17.967058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:18.056474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17701ac-2972-4f06-8d84-c880bd89a3e8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:18.065439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1917bd0-695e-44e3-aa68-6babe450925a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:18.066118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1917bd0-695e-44e3-aa68-6babe450925a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:18.066406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1917bd0-695e-44e3-aa68-6babe450925a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:18.068679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1917bd0-695e-44e3-aa68-6babe450925a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:18.091799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1917bd0-695e-44e3-aa68-6babe450925a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:18.093298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1917bd0-695e-44e3-aa68-6babe450925a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:18.112605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 857/1741] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:18.113995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade212-df68-4e80-a1cd-a806b5a66164 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:18.114288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade212-df68-4e80-a1cd-a806b5a66164 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:18.114567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ade212-df68-4e80-a1cd-a806b5a66164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:18.188213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:18.193214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-17ade212-df68-4e80-a1cd-a806b5a66164 None None] MFA Rules not processed for user `837f13aef58b455e95372c6f106d5807`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:18.202167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-17ade212-df68-4e80-a1cd-a806b5a66164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:18.206072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6639582-8cae-4a14-8e35-4bb0d009020d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:18.218674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:18.219376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:18.219661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:18.222243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:18.242793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:18.246024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:18.279817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 885/1742] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:18.281740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:18.286433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:18.335589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:18.380709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 858/1743] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:18.564404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac20c8-869f-423d-9e68-e0ed20abb4df None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:18.564723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac20c8-869f-423d-9e68-e0ed20abb4df None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:18.565373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac20c8-869f-423d-9e68-e0ed20abb4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:18.588521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb90e4-b0bc-4316-a166-e99050320daf service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:18.613092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:18.616777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:18.617407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:18.618333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] PATH_INFO: `/v3/projects/86bdf5ba521d4ac394d27005fe9f2860` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:18.620569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bdf5ba521d4ac394d27005fe9f2860)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:18.626736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:18.661048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f3ac20c8-869f-423d-9e68-e0ed20abb4df None None] MFA Rules not processed for user `bf32cd62eec04de892c3b27fbdd882a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:18.662286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f3ac20c8-869f-423d-9e68-e0ed20abb4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:18.807568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bdf5ba521d4ac394d27005fe9f2860', 'request_id': 'req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:18.842396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bdf5ba521d4ac394d27005fe9f2860', 'request_id': 'req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:18.855532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bdf5ba521d4ac394d27005fe9f2860', 'request_id': 'req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:18.888253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] The token cache is being invalidate because project 86bdf5ba521d4ac394d27005fe9f2860 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:18.888545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:18.891562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-fa6536cb-9fe7-4746-bcf7-cc2fa6f2dd20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:18.894074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 886/1744] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:18 2022] DELETE /identity/v3/projects/86bdf5ba521d4ac394d27005fe9f2860 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:18.898863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:18.899156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:18.910550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:18.911808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 859/1745] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:18.940331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:18.943787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.083632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 887/1746] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:19.092010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.093439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.093868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.133423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:19.135301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.145826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 860/1747] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:19.150852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.169166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b998e-6645-4287-8725-ee00a39d8182 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:19.178550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:19.181011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.183124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.183490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.185313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:19.186943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:19.192431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:19.223039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 888/1748] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:25:19.227806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.240252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d786a8fc-8914-4c4b-b66b-a4165a0c12fb None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:19.267058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b04ba-55d0-42c6-a837-00279b0ebc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:19.267058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b04ba-55d0-42c6-a837-00279b0ebc89 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.267372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b04ba-55d0-42c6-a837-00279b0ebc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.268166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b04ba-55d0-42c6-a837-00279b0ebc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.272761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b04ba-55d0-42c6-a837-00279b0ebc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:19.294202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b04ba-55d0-42c6-a837-00279b0ebc89 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:19.340156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 861/1749] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:19 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:19.344536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.359010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bf99f-38fe-40f0-9ab4-8372ba4fdec8 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.359554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bf99f-38fe-40f0-9ab4-8372ba4fdec8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.359977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bf99f-38fe-40f0-9ab4-8372ba4fdec8 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.364222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ee72c-dc05-4922-b97e-9443d4b1126e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:19.367383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 862/1750] 10.222.0.17 () {54 vars in 960 bytes} [Fri Jul 15 19:25:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:25:19.372502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f41f-f478-4009-986b-3e6be7c69ff5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.373814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f41f-f478-4009-986b-3e6be7c69ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.374246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d871f41f-f478-4009-986b-3e6be7c69ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.374895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:19.410099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d871f41f-f478-4009-986b-3e6be7c69ff5 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:19.413239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d871f41f-f478-4009-986b-3e6be7c69ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.515401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d346f04-697c-457c-a666-8e404b5c003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:19.518430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d346f04-697c-457c-a666-8e404b5c003d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.520515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d346f04-697c-457c-a666-8e404b5c003d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.520887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d346f04-697c-457c-a666-8e404b5c003d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.521682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d346f04-697c-457c-a666-8e404b5c003d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:19.534580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d346f04-697c-457c-a666-8e404b5c003d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:19.542399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 863/1751] 10.222.0.17 () {58 vars in 1057 bytes} [Fri Jul 15 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:19.544113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ca7a-0eeb-4357-821a-a955f165d332 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.544113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ca7a-0eeb-4357-821a-a955f165d332 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.544113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c2ca7a-0eeb-4357-821a-a955f165d332 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.545157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 864/1752] 10.222.0.17 () {54 vars in 912 bytes} [Fri Jul 15 19:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:25:19.551143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ca7a-0eeb-4357-821a-a955f165d332 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.567433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c2ca7a-0eeb-4357-821a-a955f165d332 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:19.570507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 889/1753] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:19 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:19.571961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce174-9cdb-4e91-a477-88544e91424d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.574411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce174-9cdb-4e91-a477-88544e91424d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.574704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce174-9cdb-4e91-a477-88544e91424d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.576217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:19.576953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.577318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.577693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.580030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.593656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:19.595587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:19.611082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 865/1754] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:19.612114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.617848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3c9ce174-9cdb-4e91-a477-88544e91424d None None] MFA Rules not processed for user `9f135ce75fbc4b12977cc8f009083ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:19.619112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3c9ce174-9cdb-4e91-a477-88544e91424d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.636311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6db6d9-df4e-4f4a-9dd6-f96132b8690c service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:19.650153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:19.651412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.651412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.651412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.652170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:19.654268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:19.676412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 866/1755] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:19.683398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.685516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.713047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150adc33-c349-4b23-b203-dfc8cf62cd2a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:19.724853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:19.726712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.727142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.727647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] PATH_INFO: `/v3/projects/7f1f2dfc643849329be73437304dc7d9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.732906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1f2dfc643849329be73437304dc7d9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:19.735091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:19.781313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 890/1756] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:19.782850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:19.783295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:19.783716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:19.790210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:19.825558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] MFA Rules not processed for user `01120eedaa844f24a66024df8d3b2271`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:19.827694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.856303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1f2dfc643849329be73437304dc7d9', 'request_id': 'req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:19.872306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1f2dfc643849329be73437304dc7d9', 'request_id': 'req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:19.892392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1f2dfc643849329be73437304dc7d9', 'request_id': 'req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:19.913746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] The token cache is being invalidate because project 7f1f2dfc643849329be73437304dc7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:19.914410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:19.916791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:19.919138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 867/1757] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:19 2022] DELETE /identity/v3/projects/7f1f2dfc643849329be73437304dc7d9 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:19.920842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:19.925967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:19.996550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 891/1758] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:19.997386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:20.000817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.123458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7cc97-05c4-43a4-8dbf-0f9e711334f1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:20.135818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162278eb-52a8-472a-ab66-da07a0952d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:20.137868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162278eb-52a8-472a-ab66-da07a0952d2e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.138163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162278eb-52a8-472a-ab66-da07a0952d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.139536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162278eb-52a8-472a-ab66-da07a0952d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.140344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162278eb-52a8-472a-ab66-da07a0952d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:20.142949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162278eb-52a8-472a-ab66-da07a0952d2e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:20.168782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b2c593-2f06-42d0-9da7-d8f8261583d4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:20.173343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 868/1759] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:20.176867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:20.177696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.177814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.178145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.179160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcbad7-85bd-4fef-b11f-41cfda0c53c9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.179462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcbad7-85bd-4fef-b11f-41cfda0c53c9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.179726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcbad7-85bd-4fef-b11f-41cfda0c53c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.184617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.202738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:20.211801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:20.213869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:20.221028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fddcbad7-85bd-4fef-b11f-41cfda0c53c9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:20.222433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fddcbad7-85bd-4fef-b11f-41cfda0c53c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.235630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 892/1760] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:20.237799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:20.251303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 869/1761] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:20.256417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.256715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.257172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.268619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbad205e-c86a-4e71-8b0b-cbac15c41c0e service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:20.287196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:20.289372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.289372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.289372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] PATH_INFO: `/v3/projects/cc588be6b3b342af94a6b057523b9708/users/4396c6fba1944b22a8291d0713242cd7/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.304549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:20.307122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.328657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 870/1762] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:20.330902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:20.336870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc588be6b3b342af94a6b057523b9708, user_id=4396c6fba1944b22a8291d0713242cd7, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:20.346056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:20.368270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca3913-2aed-4743-a3fd-030104dbcd34 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:20.389507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 893/1763] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:20 2022] PUT /identity/v3/projects/cc588be6b3b342af94a6b057523b9708/users/4396c6fba1944b22a8291d0713242cd7/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:20.390559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:20.396317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.540811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:20.541506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.541796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.542074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.542823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:20.543524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:20.545201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a2f58-2a6c-4608-b722-99028576f765 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:20.579418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 871/1764] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:25:20.581070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.583297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.585404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.609430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b161c5-8b04-4ecf-99a3-141e2500a2ba admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:20.629078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:20.630202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.630202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.630392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.633986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.676075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] MFA Rules not processed for user `4396c6fba1944b22a8291d0713242cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:20.681211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.703053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:20.846147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:20.846147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:20.861560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 894/1765] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:20.862229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:20.876456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:20.916119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 872/1766] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:20.918358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:20.950261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b45ed7-a793-4fba-87ae-6672ec7036db None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:20.970080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:20.971042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:20.974149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:20.974676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:20.975781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:20.979324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:21.101210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 873/1767] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:20 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:21.107052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:21.142621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623eea4-e104-44e2-a863-f0b952dd6e5b service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:21.250405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:21.251312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0036c9-18b8-49f7-9ab5-1eaabc8bc907 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:21.307076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:21.307764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:21.308052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:21.310862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:21.311614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:21.319660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:21.388975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:21.391420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:21.392432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:21.393183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:21.398917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:21.478171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 874/1768] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:21 2022] POST /identity/v3/projects => generated 373 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:21.486500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:21.600470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:21.604195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:21.630346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 895/1769] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 768 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:21.630772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:21.689063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bbd16-a6a9-4c19-95fc-584afd3c2846 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:21.829317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416743c2-2910-4ce2-8724-dc1ff8286a96 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:21.847183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:21.848294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:21.848589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:21.848915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:21.849643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:21.858685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:21.919553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:21.920633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:21.921029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:21.921413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:21.922618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:21.941086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 875/1770] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:21 2022] POST /identity/v3/users => generated 292 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:21.941673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:21.943284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:21.956225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:21.969306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e18759-24d1-44e4-9bac-3e5e4f5382c1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:21.984235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:21.985187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:21.985474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:21.985885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:21.991052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:22.001746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:22.009078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 896/1771] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:21 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:22.010901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:22.067475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 876/1772] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:22.072833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:22.073763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710058-8d27-4fa0-9380-40f461f93db3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:22.106509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:22.107212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:22.113059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:22.113300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:22.126045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:22.137048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:22.141271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2890f-168d-40f6-801b-951e46898793 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:22.156472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:22.157269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:22.158715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:22.159170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:22.159954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:22.170363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:22.176920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 897/1773] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:22.181025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:22.228272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 877/1774] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:22.228451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c05434-4638-4268-b60c-f489883db34e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:22.231987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:22.249706 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:22.252669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:22.252669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:22.252669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] PATH_INFO: `/v3/projects/c6a91b7debe64b4baab7f39db033a6dd/users/0ca2bedfd72646ec9bf6f8683bf2ca0d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:22.268928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6586cda-cda7-4c7e-9016-5c1beeda8969 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:22.296044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f3b2d-1449-4beb-8ed0-9b2187644f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:22.299412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f3b2d-1449-4beb-8ed0-9b2187644f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:22.299702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f3b2d-1449-4beb-8ed0-9b2187644f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:22.300056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f3b2d-1449-4beb-8ed0-9b2187644f89 admin admin] PATH_INFO: `/v3/projects/0c9790e81f58437ba18b1af08503f7cf/users/e78f085062f84d628373dc181acddff0/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:22.325016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a91b7debe64b4baab7f39db033a6dd, user_id=0ca2bedfd72646ec9bf6f8683bf2ca0d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:22.334129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:22.379875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f3b2d-1449-4beb-8ed0-9b2187644f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c9790e81f58437ba18b1af08503f7cf, user_id=e78f085062f84d628373dc181acddff0, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:22.383039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f3b2d-1449-4beb-8ed0-9b2187644f89 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:22.401228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 898/1775] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:22 2022] PUT /identity/v3/projects/c6a91b7debe64b4baab7f39db033a6dd/users/0ca2bedfd72646ec9bf6f8683bf2ca0d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:22.408457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:22.463735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 878/1776] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:22 2022] PUT /identity/v3/projects/0c9790e81f58437ba18b1af08503f7cf/users/e78f085062f84d628373dc181acddff0/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:22.467005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345bb6a-31bb-4916-b859-55995fca8c9f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:22.467306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345bb6a-31bb-4916-b859-55995fca8c9f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:22.467601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7345bb6a-31bb-4916-b859-55995fca8c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:22.476834 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:22.616905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7345bb6a-31bb-4916-b859-55995fca8c9f None None] MFA Rules not processed for user `e78f085062f84d628373dc181acddff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:22.622774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7345bb6a-31bb-4916-b859-55995fca8c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:22.715299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f1369-4cd9-4169-af32-ad1422bdb6a8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:22.830435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 879/1777] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:22.832445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:22.832740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:22.833024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:22.878713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] MFA Rules not processed for user `025ea412ccfa455aa84d051244902cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:22.881809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:23.063452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 880/1778] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:23.063452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:23.091124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:23.095867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.096781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.098165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.111555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:23.111892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4262492-8000-4056-8df7-13a8eb673294 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:23.118866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:23.119843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.120308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.121507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.123767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:23.126037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:23.126409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:23.185911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 899/1779] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 780 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:23.186897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:23.226431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 881/1780] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:23 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:23.231102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:23.248867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec3d1e-e354-46af-b0a7-f1f012e3bab6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:23.348493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:23.455104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2911927-577a-4ef5-a20f-fd51f798557b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:23.464664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:23.465458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.465865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.466254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.476992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:23.479243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:23.523765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344943fe-0de6-4470-8d2b-c04a83d665e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:23.527535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344943fe-0de6-4470-8d2b-c04a83d665e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.527827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344943fe-0de6-4470-8d2b-c04a83d665e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.528160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344943fe-0de6-4470-8d2b-c04a83d665e5 admin admin] PATH_INFO: `/v3/projects/c6a91b7debe64b4baab7f39db033a6dd/users/0ca2bedfd72646ec9bf6f8683bf2ca0d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.530743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344943fe-0de6-4470-8d2b-c04a83d665e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a91b7debe64b4baab7f39db033a6dd, user_id=0ca2bedfd72646ec9bf6f8683bf2ca0d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:23.535292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344943fe-0de6-4470-8d2b-c04a83d665e5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:23.546031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 882/1781] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:23 2022] POST /identity/v3/users => generated 298 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:23.548159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:23.570760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 900/1782] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:23 2022] PUT /identity/v3/projects/c6a91b7debe64b4baab7f39db033a6dd/users/0ca2bedfd72646ec9bf6f8683bf2ca0d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:23.577067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1f6429-4fea-436d-ba01-4972c0bbd30b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:23.581620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.581946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.582212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.619279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] MFA Rules not processed for user `0ca2bedfd72646ec9bf6f8683bf2ca0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:23.621590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:23.707319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:23.748577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 901/1783] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:23.756880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:23.879087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1a9dac-f4dd-439b-a8d6-84d44cafe007 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:23.885446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:23.886413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.886709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:23.886841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.887198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.887584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:23.887966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:23.888451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:23.889278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:23.898536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:23.952556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 883/1784] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:23.954192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:23.995889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa7347-934e-4be0-a101-30417d528c8f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:24.009625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:24.012335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:24.013222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:24.013987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] PATH_INFO: `/v3/projects/b56bd03d989c492a94f4c9b219b2f687/users/2fd0244cb9a44475abbaaaaf0e4c43d4/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:24.039778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:24.041756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:24.052669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 902/1785] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:24.059275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56bd03d989c492a94f4c9b219b2f687, user_id=2fd0244cb9a44475abbaaaaf0e4c43d4, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:24.063493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:24.099952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 884/1786] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:23 2022] PUT /identity/v3/projects/b56bd03d989c492a94f4c9b219b2f687/users/2fd0244cb9a44475abbaaaaf0e4c43d4/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:24.104768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:24.120300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef36f9db-d161-4632-9cad-6baaa82f0c19 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:24.243396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf12b70-fa41-4206-99c3-e830dd02a539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:24.248866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf12b70-fa41-4206-99c3-e830dd02a539 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:24.249328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf12b70-fa41-4206-99c3-e830dd02a539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:24.249959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf12b70-fa41-4206-99c3-e830dd02a539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:24.251397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf12b70-fa41-4206-99c3-e830dd02a539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:24.255162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf12b70-fa41-4206-99c3-e830dd02a539 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:24.304885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 903/1787] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:24.313176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:24.327625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a920501-d873-4586-bb7d-e49e0f3f6a68 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:24.334252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb50464-4bab-41bc-a7b5-b170e562e4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:24.337311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb50464-4bab-41bc-a7b5-b170e562e4a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:24.338390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb50464-4bab-41bc-a7b5-b170e562e4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:24.338684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb50464-4bab-41bc-a7b5-b170e562e4a9 admin admin] PATH_INFO: `/v3/projects/b56bd03d989c492a94f4c9b219b2f687/users/2fd0244cb9a44475abbaaaaf0e4c43d4/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:24.342627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb50464-4bab-41bc-a7b5-b170e562e4a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56bd03d989c492a94f4c9b219b2f687, user_id=2fd0244cb9a44475abbaaaaf0e4c43d4, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:24.346343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb50464-4bab-41bc-a7b5-b170e562e4a9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:24.395085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 885/1788] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:24 2022] PUT /identity/v3/projects/b56bd03d989c492a94f4c9b219b2f687/users/2fd0244cb9a44475abbaaaaf0e4c43d4/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:24.396920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:24.398215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:24.398753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:24.505844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] MFA Rules not processed for user `2fd0244cb9a44475abbaaaaf0e4c43d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:24.535032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:24.635695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:24.665952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:24.666187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:24.666533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:24.905443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:24.906726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:24.915096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 904/1789] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:24.978846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:24.980608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:25.180822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbae2a-da31-4385-a5c3-e05c774aa30b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:25.186555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:25.187485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:25.187924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:25.188453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:25.223978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:25.241056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:25.243071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:25.291239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 905/1790] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:25.297614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 886/1791] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:25:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 634 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:25.560095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:25.570864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441536c-7247-4e35-83f1-616a510eb690 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:25.586227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:25.586906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:25.587198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:25.591521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:25.673363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:25.675037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:25.808826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:25.809994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:25.823773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 906/1792] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:26.034816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96821455-b44d-4dd0-bb5d-999fae54ca2a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:26.046166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b49ca3-1ce6-418a-8635-6b3648046395 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:26.046877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49ca3-1ce6-418a-8635-6b3648046395 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:26.049343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49ca3-1ce6-418a-8635-6b3648046395 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:26.050087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b49ca3-1ce6-418a-8635-6b3648046395 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:26.075661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b49ca3-1ce6-418a-8635-6b3648046395 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:26.078376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b49ca3-1ce6-418a-8635-6b3648046395 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:26.091474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 887/1793] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:25:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:25:26.290440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:26.292497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:26.625314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:26.675776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e35e38-2cf3-434c-8f39-2bbd04cd8ae1 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:26.694183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:26.695490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:26.697292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:26.699292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] PATH_INFO: `/v3/users/06ac460638f149b19a74e972f7e44c29` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:26.706064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] RBAC: Authorizing `identity:delete_user(user_id=06ac460638f149b19a74e972f7e44c29)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:26.713852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:26.869488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06ac460638f149b19a74e972f7e44c29'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:26.891871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06ac460638f149b19a74e972f7e44c29'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:26.900698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06ac460638f149b19a74e972f7e44c29'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:26.901723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06ac460638f149b19a74e972f7e44c29'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:26.937573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06ac460638f149b19a74e972f7e44c29'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:26.974564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 907/1794] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:26 2022] DELETE /identity/v3/users/06ac460638f149b19a74e972f7e44c29 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:26.976646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac477c-b798-4ef1-95d1-f1b935c9f569 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:26.977123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac477c-b798-4ef1-95d1-f1b935c9f569 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:26.980228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac477c-b798-4ef1-95d1-f1b935c9f569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:27.029934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-35ac477c-b798-4ef1-95d1-f1b935c9f569 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:27.031475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-35ac477c-b798-4ef1-95d1-f1b935c9f569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:27.259148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 888/1795] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:27.432169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:27.435943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:27.622384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612416c2-91e1-491d-832d-a1ee3552535a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:27.627683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:27.628483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:27.628796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:27.629234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:27.647507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:27.649281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:27.668922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 908/1796] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:29.134929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:29.135443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:29.135525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:29.175357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] MFA Rules not processed for user `bf32cd62eec04de892c3b27fbdd882a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:29.180865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:29.351432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 889/1797] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:29.633410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:29.635803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:29.664471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e092e04-cfab-4050-a7a8-e763f4d64fc9 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:29.673540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:29.673540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:29.673871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:29.674181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] PATH_INFO: `/v3/projects/3934c717927341e785dd540b2131c6fb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:29.675714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3934c717927341e785dd540b2131c6fb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:29.677781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:29.759999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3934c717927341e785dd540b2131c6fb', 'request_id': 'req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:29.797170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3934c717927341e785dd540b2131c6fb', 'request_id': 'req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:29.813426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3934c717927341e785dd540b2131c6fb', 'request_id': 'req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:29.832636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:29.840010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] The token cache is being invalidate because project 3934c717927341e785dd540b2131c6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:29.843391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:29.844580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:29.851742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 909/1798] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:29 2022] DELETE /identity/v3/projects/3934c717927341e785dd540b2131c6fb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:29.854581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:29.871259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470882d-a590-4db1-9217-720ea20a7fdd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:29.881862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:29.996331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:29.997141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:29.997433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:29.997716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] PATH_INFO: `/v3/users/81680a6f5b8d476daa4781d73c2f6908` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:30.000877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81680a6f5b8d476daa4781d73c2f6908)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:30.002866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:30.118768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81680a6f5b8d476daa4781d73c2f6908'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.136678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81680a6f5b8d476daa4781d73c2f6908'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.138316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81680a6f5b8d476daa4781d73c2f6908'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.139109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81680a6f5b8d476daa4781d73c2f6908'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.155734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81680a6f5b8d476daa4781d73c2f6908'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.184197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 890/1799] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:29 2022] DELETE /identity/v3/users/81680a6f5b8d476daa4781d73c2f6908 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:30.778638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:30.782072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:30.801889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b7132-2bf8-433e-a4af-7c1e2c60b58d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:30.881516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:30.882685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:30.883101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:30.883622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] PATH_INFO: `/v3/projects/57000d0f3e5b478ba26bb77a15d24c00` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:30.884962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57000d0f3e5b478ba26bb77a15d24c00)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:30.886985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:30.958174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57000d0f3e5b478ba26bb77a15d24c00', 'request_id': 'req-3691bd8a-03ad-4b16-8591-b6fe6966e979'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.977711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57000d0f3e5b478ba26bb77a15d24c00', 'request_id': 'req-3691bd8a-03ad-4b16-8591-b6fe6966e979'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:30.985467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57000d0f3e5b478ba26bb77a15d24c00', 'request_id': 'req-3691bd8a-03ad-4b16-8591-b6fe6966e979'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.000616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] The token cache is being invalidate because project 57000d0f3e5b478ba26bb77a15d24c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:31.000830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.001707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.003748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 910/1800] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:30 2022] DELETE /identity/v3/projects/57000d0f3e5b478ba26bb77a15d24c00 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:31.008765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:31.010189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:31.027711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703b64ac-752c-41ae-9935-892d81c1d72d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:31.099047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:31.100596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:31.100882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:31.101162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] PATH_INFO: `/v3/users/a71f59b584104ff1bb0027d80e999592` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:31.102523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71f59b584104ff1bb0027d80e999592)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:31.104527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:31.159034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:31.197841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71f59b584104ff1bb0027d80e999592'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.213296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71f59b584104ff1bb0027d80e999592'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.214532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71f59b584104ff1bb0027d80e999592'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.215554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71f59b584104ff1bb0027d80e999592'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.228388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f64e9f0c-4969-4e0e-b3bd-657de04e1194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71f59b584104ff1bb0027d80e999592'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.263176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 891/1801] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:31 2022] DELETE /identity/v3/users/a71f59b584104ff1bb0027d80e999592 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:31.779381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:31.782054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:31.797555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3691bd8a-03ad-4b16-8591-b6fe6966e979 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:31.864669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:31.865904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:31.866191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:31.866612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] PATH_INFO: `/v3/projects/8c931d0876e8497ab0850a15c2d3e030` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:31.867868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c931d0876e8497ab0850a15c2d3e030)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:31.870195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:31.915474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:31.935063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c931d0876e8497ab0850a15c2d3e030', 'request_id': 'req-13a976eb-237c-4a26-aa3f-e218484e5911'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.947090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c931d0876e8497ab0850a15c2d3e030', 'request_id': 'req-13a976eb-237c-4a26-aa3f-e218484e5911'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.958944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c931d0876e8497ab0850a15c2d3e030', 'request_id': 'req-13a976eb-237c-4a26-aa3f-e218484e5911'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.988047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] The token cache is being invalidate because project 8c931d0876e8497ab0850a15c2d3e030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:31.988352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.989062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13a976eb-237c-4a26-aa3f-e218484e5911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:31.992115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 911/1802] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:31 2022] DELETE /identity/v3/projects/8c931d0876e8497ab0850a15c2d3e030 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:31.999008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:31.999301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:31.999831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.019749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:32.020804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:32.088551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 892/1803] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:32.097029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.097262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.097546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.115883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:32.117117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:32.125207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 912/1804] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:32.129332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.139734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ac25-cfe1-4a50-a5a0-889f8effc26c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.144291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.144944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.145230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.145508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.146233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:32.147325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.148969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.160646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 893/1805] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:25:32.164886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.174032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aec0df4-4e13-4c01-9f72-0752db75b2ba None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.179362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.180009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.180326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.180604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.181293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.183201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.208352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 913/1806] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:32 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:32.212930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.222747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7b0bb-3c18-4eed-86c4-5c39181eb03c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.278686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.279357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.279671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.279948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.280683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.282596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.305281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 894/1807] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:32 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:32.306995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.317485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ac3d4-513c-4ed0-b1eb-3d84d327202f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.322404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.323078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.323363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.323642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.324418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.326019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.337925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 914/1808] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:32.341837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.359602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebe909b-7927-469f-8e09-7c1c6a2111ee admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.371642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db4d0e-d9c5-4511-aedc-f64aad914d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.373437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4d0e-d9c5-4511-aedc-f64aad914d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.373837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4d0e-d9c5-4511-aedc-f64aad914d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.374589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db4d0e-d9c5-4511-aedc-f64aad914d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.375518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db4d0e-d9c5-4511-aedc-f64aad914d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.377788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db4d0e-d9c5-4511-aedc-f64aad914d97 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.389984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 895/1809] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:32.401896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.416569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b69cd-93f6-453d-a824-c95f289053f6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.430685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.431958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.432354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.432892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] PATH_INFO: `/v3/projects/37e6aa46444c4895ade470d83dec7c9f/users/0605f961c6f24d1fbaf84d964590f1e3/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.459628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e6aa46444c4895ade470d83dec7c9f, user_id=0605f961c6f24d1fbaf84d964590f1e3, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.462761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.477901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 915/1810] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:32 2022] PUT /identity/v3/projects/37e6aa46444c4895ade470d83dec7c9f/users/0605f961c6f24d1fbaf84d964590f1e3/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:32.492602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.492924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.493214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.517488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:32.527365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] MFA Rules not processed for user `0605f961c6f24d1fbaf84d964590f1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:32.530448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:32.624874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 896/1811] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:32.629372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.727835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c94aa1-a9d4-4760-8765-c2923155d34c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.732823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.733486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.733766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.734040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.734744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.744924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.778243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 916/1812] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:32 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:32.782412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.794944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8ac2c-5016-4de0-95e2-939e16c93ab8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.858199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.859502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.859793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.860075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.861260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.863293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.896117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 897/1813] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:32 2022] POST /identity/v3/users => generated 298 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:32.898554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.909207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a9843e-8472-41e1-bee1-20805ca6fe28 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.914902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.915578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.915863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.916172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.916913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.918529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.931202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 917/1814] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:32.934989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:32.945673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b76c4-feac-467f-a2e1-02a2a9e0d2df admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:32.950974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:32.951899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:32.952232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:32.952507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] PATH_INFO: `/v3/projects/880846e52f444e86a37169042144d3cc/users/5408b978055e497cbefa0ceaec65e468/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:32.969993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=880846e52f444e86a37169042144d3cc, user_id=5408b978055e497cbefa0ceaec65e468, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:32.972984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:32.988709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 898/1815] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:32 2022] PUT /identity/v3/projects/880846e52f444e86a37169042144d3cc/users/5408b978055e497cbefa0ceaec65e468/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:32.992768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:33.005747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73833b-b8ff-43c3-a020-410fcb86a668 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:33.034539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:33.075822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4d7d61-8861-42f0-824b-f89f2d0d6437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:33.076816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d7d61-8861-42f0-824b-f89f2d0d6437 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:33.077103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d7d61-8861-42f0-824b-f89f2d0d6437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:33.077385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d7d61-8861-42f0-824b-f89f2d0d6437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:33.078128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d7d61-8861-42f0-824b-f89f2d0d6437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:33.079762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4d7d61-8861-42f0-824b-f89f2d0d6437 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:33.096658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 918/1816] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:33.099396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:33.111415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a122b1a2-278e-4ded-ba28-105ae98bcc68 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:33.116332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:33.117305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:33.117593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:33.117876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] PATH_INFO: `/v3/projects/880846e52f444e86a37169042144d3cc/users/5408b978055e497cbefa0ceaec65e468/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:33.119953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=880846e52f444e86a37169042144d3cc, user_id=5408b978055e497cbefa0ceaec65e468, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:33.122841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:33.144527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 899/1817] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:33 2022] PUT /identity/v3/projects/880846e52f444e86a37169042144d3cc/users/5408b978055e497cbefa0ceaec65e468/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:33.147031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:33.147369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:33.147654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:33.190473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] MFA Rules not processed for user `5408b978055e497cbefa0ceaec65e468`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:33.192009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:33.279073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 919/1818] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:33.291851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:33.293545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:33.356804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c79f53-656e-4a85-8259-8a20f8b29eae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:33.360502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:33.361197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:33.361487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:33.361769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:33.371126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:33.374256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:33.376456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:33.386562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 900/1819] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:33.472274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:33.473680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:33.485857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08c3da-e86d-47ce-9c73-2d9df4ffc70d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:33.489975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31660a6d-5394-426b-8571-36700c7a8129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:33.490747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660a6d-5394-426b-8571-36700c7a8129 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:33.491107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660a6d-5394-426b-8571-36700c7a8129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:33.491399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31660a6d-5394-426b-8571-36700c7a8129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:33.550279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31660a6d-5394-426b-8571-36700c7a8129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:33.551727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31660a6d-5394-426b-8571-36700c7a8129 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:33.561587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 920/1820] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:33.980034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:34.076123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f5ba-8334-4256-9268-d5e3af06adb7 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:34.081056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:34.081381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:34.081695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:34.081974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] PATH_INFO: `/v3/users/0605f961c6f24d1fbaf84d964590f1e3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:34.083200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0605f961c6f24d1fbaf84d964590f1e3)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:34.085263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:34.138579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0605f961c6f24d1fbaf84d964590f1e3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.146615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0605f961c6f24d1fbaf84d964590f1e3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.152251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0605f961c6f24d1fbaf84d964590f1e3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.153045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0605f961c6f24d1fbaf84d964590f1e3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.160779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0605f961c6f24d1fbaf84d964590f1e3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.181236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 901/1821] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:33 2022] DELETE /identity/v3/users/0605f961c6f24d1fbaf84d964590f1e3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:34.185948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:34.186792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:34.187073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:34.209118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:34.210683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:34.288738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:34.335780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 921/1822] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:34.346284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:34.348445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:34.461075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e185e-0892-447e-b71a-549209a8210a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:34.466594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:34.466594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:34.466594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:34.466906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:34.479586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:34.481825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:34.496801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 902/1823] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:34.518767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:34.520599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:34.636954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae821f4-e3b9-4019-8a86-cef787f4779a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:34.640649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:34.641345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:34.641626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:34.641951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:34.643780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:34.690835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:34.692324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:34.716462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e933d5-c68b-4184-b3a8-5242360921fc service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:34.732176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:34.733670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a4253-0763-4b9e-b494-6359f0264616 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:34.733925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:34.734846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:34.735127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:34.735400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] PATH_INFO: `/v3/users/e78f085062f84d628373dc181acddff0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:34.736726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e78f085062f84d628373dc181acddff0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:34.738691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:34.762977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 922/1824] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:34.765820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:34.835838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e78f085062f84d628373dc181acddff0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.853023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e78f085062f84d628373dc181acddff0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.854292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e78f085062f84d628373dc181acddff0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.855229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e78f085062f84d628373dc181acddff0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.866423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e78f085062f84d628373dc181acddff0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:34.893428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 903/1825] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:34 2022] DELETE /identity/v3/users/e78f085062f84d628373dc181acddff0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:34.895102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:34.895335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:34.898925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:34.954064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:34.956543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:35.185333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 923/1826] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:35.195053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:35.196821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:35.241811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:35.242886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:35.263222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc770-33f0-4a92-b8e7-cb22e76c10cd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:35.270087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:35.270892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:35.271226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:35.272336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] PATH_INFO: `/v3/projects/37e6aa46444c4895ade470d83dec7c9f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:35.272991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e6aa46444c4895ade470d83dec7c9f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:35.275324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:35.362124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:35.395119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e6aa46444c4895ade470d83dec7c9f', 'request_id': 'req-f3008626-cc56-4854-9616-1625e72b7b03'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.403288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaed5b8f-df64-45ec-a2be-33c32c5ceef1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:35.411772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e6aa46444c4895ade470d83dec7c9f', 'request_id': 'req-f3008626-cc56-4854-9616-1625e72b7b03'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.430917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e6aa46444c4895ade470d83dec7c9f', 'request_id': 'req-f3008626-cc56-4854-9616-1625e72b7b03'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.463925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] The token cache is being invalidate because project 37e6aa46444c4895ade470d83dec7c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:35.464256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.465020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.471848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 924/1827] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:35 2022] DELETE /identity/v3/projects/37e6aa46444c4895ade470d83dec7c9f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:35.477489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:35.479262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:35.528199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:35.528897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:35.529371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:35.529650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:35.532630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:35.608547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3008626-cc56-4854-9616-1625e72b7b03 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:35.626117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:35.631252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:35.631563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:35.631809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] PATH_INFO: `/v3/users/5408b978055e497cbefa0ceaec65e468` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:35.634534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5408b978055e497cbefa0ceaec65e468)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:35.636891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:35.654243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:35.657769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:35.660652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:35.674886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 904/1828] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:35.737645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5408b978055e497cbefa0ceaec65e468'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.758109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5408b978055e497cbefa0ceaec65e468'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.758871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5408b978055e497cbefa0ceaec65e468'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.759569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5408b978055e497cbefa0ceaec65e468'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.778420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5408b978055e497cbefa0ceaec65e468'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:35.800154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 925/1829] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:35 2022] DELETE /identity/v3/users/5408b978055e497cbefa0ceaec65e468 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:36.223229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:36.232134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:36.232307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:36.232307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:36.251732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02163a-c5fa-4332-9fd1-3c5014fac322 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:36.332029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:36.359324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a9a4-927f-4473-9ff8-d22576eed1b2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:36.371801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:36.373578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.373742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.374084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] PATH_INFO: `/v3/projects/0c9790e81f58437ba18b1af08503f7cf` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.377912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c9790e81f58437ba18b1af08503f7cf)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:36.379275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:36.380204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.380496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.380847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] PATH_INFO: `/v3/projects/880846e52f444e86a37169042144d3cc` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.382787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=880846e52f444e86a37169042144d3cc)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:36.383279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:36.384784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:36.490354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880846e52f444e86a37169042144d3cc', 'request_id': 'req-5f7fa541-aaf9-45df-9a04-ede3f2208336'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.505395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880846e52f444e86a37169042144d3cc', 'request_id': 'req-5f7fa541-aaf9-45df-9a04-ede3f2208336'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.511227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c9790e81f58437ba18b1af08503f7cf', 'request_id': 'req-ab0aa36d-5b12-495f-826b-28389ab76ecc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.517806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880846e52f444e86a37169042144d3cc', 'request_id': 'req-5f7fa541-aaf9-45df-9a04-ede3f2208336'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.529189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c9790e81f58437ba18b1af08503f7cf', 'request_id': 'req-ab0aa36d-5b12-495f-826b-28389ab76ecc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.538466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] The token cache is being invalidate because project 880846e52f444e86a37169042144d3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:36.538756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.540264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f7fa541-aaf9-45df-9a04-ede3f2208336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.542867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 926/1830] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:36 2022] DELETE /identity/v3/projects/880846e52f444e86a37169042144d3cc => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:36.550781 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba395c-e006-4bdf-8285-f15decbd6073 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.551550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba395c-e006-4bdf-8285-f15decbd6073 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.551985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c9790e81f58437ba18b1af08503f7cf', 'request_id': 'req-ab0aa36d-5b12-495f-826b-28389ab76ecc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.552158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba395c-e006-4bdf-8285-f15decbd6073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.586184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d8ba395c-e006-4bdf-8285-f15decbd6073 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:36.588449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] The token cache is being invalidate because project 0c9790e81f58437ba18b1af08503f7cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:36.588746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d8ba395c-e006-4bdf-8285-f15decbd6073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:36.588886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.593436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.599060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 905/1831] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:36 2022] DELETE /identity/v3/projects/0c9790e81f58437ba18b1af08503f7cf => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:36.604194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:36.606208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:36.736831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 927/1832] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:36.747700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.748259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.748539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.749160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0aa36d-5b12-495f-826b-28389ab76ecc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:36.754136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:36.755091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.755453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.756387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] PATH_INFO: `/v3/users/2fd0244cb9a44475abbaaaaf0e4c43d4` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.757741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd0244cb9a44475abbaaaaf0e4c43d4)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:36.759760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:36.774134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:36.775363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:36.786361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 928/1833] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:36.788483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:36.798020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283de051-5297-4e9a-bbe9-24740ab4d8d4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:36.800753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:36.802822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:36.803499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.803753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.804028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.804784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:36.805085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:36.806681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:36.818950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 929/1834] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:25:36.821129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd0244cb9a44475abbaaaaf0e4c43d4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.823425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:36.828800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd0244cb9a44475abbaaaaf0e4c43d4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.829556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd0244cb9a44475abbaaaaf0e4c43d4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.830230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd0244cb9a44475abbaaaaf0e4c43d4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.835268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48074cb6-9ff1-4bde-89de-b6fd63d63dab admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:36.838002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd0244cb9a44475abbaaaaf0e4c43d4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:36.839922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:36.840603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:36.840890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:36.841171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:36.841859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:36.843783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:36.857565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 906/1835] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:36 2022] DELETE /identity/v3/users/2fd0244cb9a44475abbaaaaf0e4c43d4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:36.871585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 930/1836] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:36 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:36.876071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:36.878215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:36.953091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c045ce80-b942-43aa-a6dd-d5f85ea305b4 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.015000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.015709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.015993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.016316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.017015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.018971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.043355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 907/1837] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:36 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:37.044650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.055591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2e59f-0242-49a8-a981-e62399e43ecc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.061086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.061737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.062022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.062811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.063174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.064837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.075999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 931/1838] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:37.079966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.090783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdcb5a-1ef1-431e-973a-69a023c5f40d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.095313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9657c6ca-82fd-4485-b1ec-804671af4f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.096070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c6ca-82fd-4485-b1ec-804671af4f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.096402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c6ca-82fd-4485-b1ec-804671af4f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.096684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9657c6ca-82fd-4485-b1ec-804671af4f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.097494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657c6ca-82fd-4485-b1ec-804671af4f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.099115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9657c6ca-82fd-4485-b1ec-804671af4f30 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.118136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 908/1839] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:37.122316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.133667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea986f-ab2d-465b-b30b-0a5fc39b4563 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.138428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.140796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.140796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.140796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] PATH_INFO: `/v3/projects/9fd48b64c7df4dc4a6a7b9dccf381b99/users/3e514bae272645bab60683f743364911/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.156336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd48b64c7df4dc4a6a7b9dccf381b99, user_id=3e514bae272645bab60683f743364911, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.159006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.163015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:37.188137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 932/1840] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:37 2022] PUT /identity/v3/projects/9fd48b64c7df4dc4a6a7b9dccf381b99/users/3e514bae272645bab60683f743364911/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:37.192469 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.192766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.193085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.219629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] MFA Rules not processed for user `3e514bae272645bab60683f743364911`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:37.221097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:37.242483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:37.278519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.281020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:37.317777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 909/1841] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:37.322654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.353355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788313f5-8f96-4aa6-aa64-525cadecc9a7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.358745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.359791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.360241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.360757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] PATH_INFO: `/v3/projects/b56bd03d989c492a94f4c9b219b2f687` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.362152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56bd03d989c492a94f4c9b219b2f687)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.364224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.392221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7cd06-2b44-4b43-b2b0-84373ef48ac5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.397564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.398277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.398588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.398925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.399730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.401669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.416383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56bd03d989c492a94f4c9b219b2f687', 'request_id': 'req-de950eed-cffa-4fc9-ab65-d9f20aeea44a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:37.423598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56bd03d989c492a94f4c9b219b2f687', 'request_id': 'req-de950eed-cffa-4fc9-ab65-d9f20aeea44a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:37.428416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 910/1842] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:37.430707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56bd03d989c492a94f4c9b219b2f687', 'request_id': 'req-de950eed-cffa-4fc9-ab65-d9f20aeea44a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:37.432378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.441459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] The token cache is being invalidate because project b56bd03d989c492a94f4c9b219b2f687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:37.441743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:37.442412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-de950eed-cffa-4fc9-ab65-d9f20aeea44a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:37.443999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 933/1843] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:37 2022] DELETE /identity/v3/projects/b56bd03d989c492a94f4c9b219b2f687 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:37.448505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed81ed7-98ab-4f2a-a3c0-8f2ee6696651 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.448791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed81ed7-98ab-4f2a-a3c0-8f2ee6696651 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.449068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed81ed7-98ab-4f2a-a3c0-8f2ee6696651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.469133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fed81ed7-98ab-4f2a-a3c0-8f2ee6696651 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:37.470334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fed81ed7-98ab-4f2a-a3c0-8f2ee6696651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:37.506422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1909a5d-6edd-4522-a666-4413874da011 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.512346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.512881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.513387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.513499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.514334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.516637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.542757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 911/1844] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:37.546793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.548728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:37.554365 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 934/1845] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:37.558311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.558760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.559046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.571217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ba374-bdde-480c-a6d6-273dc7f1ec39 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.581458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.582142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.582479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.582773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.583667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.585565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.589428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:37.590653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:37.596630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 912/1846] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:37.600676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.602259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 935/1847] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:37.608073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.613349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79656dbe-763d-4e26-934e-879da8e6e8b0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.618045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb0fcf-53ff-45b9-b26a-ab1163b8a88e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.618698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.619356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.619638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.619914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.620698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.622297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.623252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.624022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.624394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.624738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.625569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:37.625880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.627559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.634694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 913/1848] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:37.638767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.643369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 936/1849] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:25:37.644150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.648681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfec0b2-48f4-4cf0-91d3-6eb0782ae157 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.653368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.653846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3dd9d5-26d2-4e19-b0df-a0b76d323082 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.654242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.654523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.654802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] PATH_INFO: `/v3/projects/3c0d7b13fe5c4e8da17718533e482c01/users/0814c9b009c045fc8a87c2af98ae0d06/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.658965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a5cf9-319e-4cf3-988d-2df35c5a6a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.659601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5cf9-319e-4cf3-988d-2df35c5a6a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.659883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5cf9-319e-4cf3-988d-2df35c5a6a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.660194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a5cf9-319e-4cf3-988d-2df35c5a6a82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.660907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5cf9-319e-4cf3-988d-2df35c5a6a82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.662891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a5cf9-319e-4cf3-988d-2df35c5a6a82 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.666212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:37.666346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:37.671197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0d7b13fe5c4e8da17718533e482c01, user_id=0814c9b009c045fc8a87c2af98ae0d06, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.674546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.695186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 914/1850] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:37 2022] PUT /identity/v3/projects/3c0d7b13fe5c4e8da17718533e482c01/users/0814c9b009c045fc8a87c2af98ae0d06/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:37.700597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 937/1851] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:37.704108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.705488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.705780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.706056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.733398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] MFA Rules not processed for user `0814c9b009c045fc8a87c2af98ae0d06`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:37.735543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:37.782444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e28de2f-7917-4d24-b399-d4faf1a0b36f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.831817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 938/1852] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:37.836250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.901157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.902318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.902715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.903093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.903929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.906031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.927073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80cdda3-f511-4af4-aa17-d7f99f049646 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.944905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 915/1853] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:37.949846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.950952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.951240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.951512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.952796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.954935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:37.960555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:37.975739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d0f8-271a-4db4-921f-e69f7a4b9901 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:37.983239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:37.984154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:37.984581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:37.985137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:37.986240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:37.988102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.013944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 916/1854] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:38.014859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 939/1855] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:37 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:38.022564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.025597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.036625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8765899b-7476-4d90-ba18-ad6bec19aa8e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.107178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.107869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.108187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.108486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.109465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.111083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.123959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 917/1856] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:38.128358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.138400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb5d1e-9f99-4bd8-9f71-d4aa06ba24c2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.141134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1316bc81-a41f-4a35-837e-416d5d5e14d3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.143351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.144130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.144507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.144850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.145614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.148289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946abef4-9ead-4c70-92f6-4f926c767033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.149281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.149851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946abef4-9ead-4c70-92f6-4f926c767033 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.150190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946abef4-9ead-4c70-92f6-4f926c767033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.150497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946abef4-9ead-4c70-92f6-4f926c767033 admin admin] PATH_INFO: `/v3/projects/ece4a0b4aff4411f9dfd8f615160140e/users/a3d0b5d1659f4b8eb99b572a67b3d1b1/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.156203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:38.163505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:38.179238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946abef4-9ead-4c70-92f6-4f926c767033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece4a0b4aff4411f9dfd8f615160140e, user_id=a3d0b5d1659f4b8eb99b572a67b3d1b1, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.182724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946abef4-9ead-4c70-92f6-4f926c767033 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.183558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 940/1857] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:38 2022] POST /identity/v3/users => generated 298 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:38.187593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.200740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f192cc7-febb-4b13-a035-42130bc5f556 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.201374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 918/1858] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:38 2022] PUT /identity/v3/projects/ece4a0b4aff4411f9dfd8f615160140e/users/a3d0b5d1659f4b8eb99b572a67b3d1b1/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:38.211526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.211824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.212127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.248826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] MFA Rules not processed for user `a3d0b5d1659f4b8eb99b572a67b3d1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:38.250207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:38.266051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.266884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.267173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.267447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.268409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.271670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.286082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 941/1859] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:38.290217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.301009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d95b4-e0c3-4cce-87d2-ac795237c1f1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.307284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.308207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.308478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.308755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] PATH_INFO: `/v3/projects/34ecd63b6a8c4c42b93b2cdd6eb3228b/users/69879ba6c71a452caf86ffcceb87a12c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.325843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ecd63b6a8c4c42b93b2cdd6eb3228b, user_id=69879ba6c71a452caf86ffcceb87a12c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.328889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.344448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 919/1860] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:38.348267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 942/1861] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:38 2022] PUT /identity/v3/projects/34ecd63b6a8c4c42b93b2cdd6eb3228b/users/69879ba6c71a452caf86ffcceb87a12c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:38.348348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.349093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.358582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c603e4f6-5ff4-4831-876d-14e765f16da0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.442481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.443221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.444293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.444587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.445307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.446002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd922c1-b8de-40f3-ad42-cb4c4b60e9f4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.447264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.450797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.451494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.451796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.452049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.452815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.454446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.471434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 943/1862] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:38.471434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.481985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a160780f-80bd-4628-875f-1c327ba3eb40 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.486701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69490af1-35e9-4184-87c5-959b810d2a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.487735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69490af1-35e9-4184-87c5-959b810d2a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.487924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69490af1-35e9-4184-87c5-959b810d2a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.488281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69490af1-35e9-4184-87c5-959b810d2a9b admin admin] PATH_INFO: `/v3/projects/34ecd63b6a8c4c42b93b2cdd6eb3228b/users/69879ba6c71a452caf86ffcceb87a12c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.491007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69490af1-35e9-4184-87c5-959b810d2a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=34ecd63b6a8c4c42b93b2cdd6eb3228b, user_id=69879ba6c71a452caf86ffcceb87a12c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.493964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69490af1-35e9-4184-87c5-959b810d2a9b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.497823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 920/1863] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:38 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:38.502130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.507232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 944/1864] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:38 2022] PUT /identity/v3/projects/34ecd63b6a8c4c42b93b2cdd6eb3228b/users/69879ba6c71a452caf86ffcceb87a12c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:38.517441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.517673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.517957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.549863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] MFA Rules not processed for user `69879ba6c71a452caf86ffcceb87a12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:38.551212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:38.582570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad31171-d680-4c7e-92c7-3816f4fe6107 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.593783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:38.603401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:38.665625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 945/1865] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:38.678111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.679570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:38.702003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.702698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.703068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.703354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.704374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.708348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.737977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 921/1866] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:38 2022] POST /identity/v3/users => generated 296 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:38.742067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.759972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970dba3-5bc3-48c6-b220-6039732ff8e8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.765208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.768193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.768193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.768193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.768193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.770300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.781731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1e617-9a2b-41ad-b193-a43ed2083007 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.784384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 922/1867] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:38.788611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.793011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.794558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.794933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.795331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.798190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:38.803757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178ca1a-d474-4a6a-bc46-756d65752e5a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:38.828329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:38.829295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:38.829840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:38.830191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] PATH_INFO: `/v3/projects/2eeac6b92b46404b9cba50052487cab6/users/3773cd8ef747479390267b51831ddc8d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:38.852343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eeac6b92b46404b9cba50052487cab6, user_id=3773cd8ef747479390267b51831ddc8d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.855304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.874676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 923/1868] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:38 2022] PUT /identity/v3/projects/2eeac6b92b46404b9cba50052487cab6/users/3773cd8ef747479390267b51831ddc8d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:38.879255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:38.890898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:38.892652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:38.905257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 946/1869] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:39.016136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18949cce-be7b-4f28-a569-ff5fe39a117f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:39.021620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2893e-8abf-447f-a19d-2433ec145769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:39.022310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2893e-8abf-447f-a19d-2433ec145769 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:39.022597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2893e-8abf-447f-a19d-2433ec145769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:39.022871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2893e-8abf-447f-a19d-2433ec145769 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:39.034219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2893e-8abf-447f-a19d-2433ec145769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:39.036243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2893e-8abf-447f-a19d-2433ec145769 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:39.054080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 924/1870] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:39.057982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:39.085260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047bb9e-1567-430b-b0b8-87b8714fbba4 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:39.092877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:39.094493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:39.094807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:39.095274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] PATH_INFO: `/v3/projects/2eeac6b92b46404b9cba50052487cab6/users/3773cd8ef747479390267b51831ddc8d/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:39.098025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eeac6b92b46404b9cba50052487cab6, user_id=3773cd8ef747479390267b51831ddc8d, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:39.107943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:39.144435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 947/1871] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:39 2022] PUT /identity/v3/projects/2eeac6b92b46404b9cba50052487cab6/users/3773cd8ef747479390267b51831ddc8d/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:39.144972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:39.145346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:39.145634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:39.212001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] MFA Rules not processed for user `3773cd8ef747479390267b51831ddc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:39.218993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:39.304167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:39.363740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 925/1872] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:39.383792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:39.523329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d5e27-d09d-406f-8ae7-4cdbfae7406b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:39.526892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f79608-4eb3-4b9f-8004-fbf13e354363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:39.527539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79608-4eb3-4b9f-8004-fbf13e354363 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:39.527823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79608-4eb3-4b9f-8004-fbf13e354363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:39.528119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f79608-4eb3-4b9f-8004-fbf13e354363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:39.538007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79608-4eb3-4b9f-8004-fbf13e354363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:39.539503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f79608-4eb3-4b9f-8004-fbf13e354363 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:39.549424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 948/1873] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:39.948999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:40.014447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7781c-7175-4e06-b733-c218b2a305e7 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:40.019273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:40.020207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:40.020496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:40.020775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] PATH_INFO: `/v3/users/a3d0b5d1659f4b8eb99b572a67b3d1b1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:40.022122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d0b5d1659f4b8eb99b572a67b3d1b1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:40.024154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:40.088947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d0b5d1659f4b8eb99b572a67b3d1b1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:40.098229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d0b5d1659f4b8eb99b572a67b3d1b1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:40.098964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d0b5d1659f4b8eb99b572a67b3d1b1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:40.100235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d0b5d1659f4b8eb99b572a67b3d1b1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:40.107697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d0b5d1659f4b8eb99b572a67b3d1b1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:40.125828 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 926/1874] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:39 2022] DELETE /identity/v3/users/a3d0b5d1659f4b8eb99b572a67b3d1b1 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:40.130392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:40.130798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:40.131157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:40.150309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:40.151695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:40.165172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:40.227361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 949/1875] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:40.238612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:40.240927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:40.329258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a11b51-27a8-4516-873d-a6d043de60dd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:40.345698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:40.347479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:40.347866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:40.348265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:40.363278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:40.365625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:40.382883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 927/1876] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:41.051637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:41.054036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:41.068922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e86033-22ff-484f-b261-fe1936d51ffd None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:41.075959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:41.076871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:41.077158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:41.077447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] PATH_INFO: `/v3/projects/ece4a0b4aff4411f9dfd8f615160140e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:41.078646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece4a0b4aff4411f9dfd8f615160140e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:41.080635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:41.143066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece4a0b4aff4411f9dfd8f615160140e', 'request_id': 'req-13b62447-bc5a-473b-bc6e-996413c368a5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.153471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece4a0b4aff4411f9dfd8f615160140e', 'request_id': 'req-13b62447-bc5a-473b-bc6e-996413c368a5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.165133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece4a0b4aff4411f9dfd8f615160140e', 'request_id': 'req-13b62447-bc5a-473b-bc6e-996413c368a5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.184002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] The token cache is being invalidate because project ece4a0b4aff4411f9dfd8f615160140e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:41.197454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.198667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.201913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 950/1877] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:41 2022] DELETE /identity/v3/projects/ece4a0b4aff4411f9dfd8f615160140e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:41.204969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:41.216446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93a00a2-1917-40f2-89d0-f83bb940fd78 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:41.220524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:41.295364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:41.296346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:41.296629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:41.296901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] PATH_INFO: `/v3/users/3773cd8ef747479390267b51831ddc8d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:41.298086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3773cd8ef747479390267b51831ddc8d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:41.300008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:41.348827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3773cd8ef747479390267b51831ddc8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.356805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3773cd8ef747479390267b51831ddc8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.357523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3773cd8ef747479390267b51831ddc8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.358170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3773cd8ef747479390267b51831ddc8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.365525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ac2daca7-e41f-474b-a7df-da0096ea662a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3773cd8ef747479390267b51831ddc8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.383894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 928/1878] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:41 2022] DELETE /identity/v3/users/3773cd8ef747479390267b51831ddc8d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:41.677669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:41.679749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:41.694595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b62447-bc5a-473b-bc6e-996413c368a5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:41.770724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:41.772323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:41.772613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:41.772896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] PATH_INFO: `/v3/projects/2eeac6b92b46404b9cba50052487cab6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:41.774247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eeac6b92b46404b9cba50052487cab6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:41.782226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:41.829923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:41.839288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eeac6b92b46404b9cba50052487cab6', 'request_id': 'req-240525de-0b1a-4e48-a274-39fd48ebb5b6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.847004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eeac6b92b46404b9cba50052487cab6', 'request_id': 'req-240525de-0b1a-4e48-a274-39fd48ebb5b6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.854051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eeac6b92b46404b9cba50052487cab6', 'request_id': 'req-240525de-0b1a-4e48-a274-39fd48ebb5b6'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.865698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] The token cache is being invalidate because project 2eeac6b92b46404b9cba50052487cab6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:41.865984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.866637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-240525de-0b1a-4e48-a274-39fd48ebb5b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:41.868599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 951/1879] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:41 2022] DELETE /identity/v3/projects/2eeac6b92b46404b9cba50052487cab6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:41.874644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:41.875031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:41.875387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:41.896815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:41.898142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:41.992901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 929/1880] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:42.001666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.002071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.002465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.041607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:42.044870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:42.074345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 952/1881] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:42.081533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.092520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c723e-41a3-4574-9270-a5f070a336aa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.097456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.098186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.098477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.098756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.099536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:25:42.099844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.101719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.140556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 930/1882] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:25:42.141435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.151422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f9d35-0e74-4d86-b304-53b0f63f6ac0 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.157654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.158336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.158618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.158890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.159593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.161637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.200954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 953/1883] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:42 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:42.204845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.215323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5353d-026e-4107-bb84-ecbf03c6cfb2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.230270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:42.285407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.286099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.286391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.286720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.287607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.294043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.317517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 931/1884] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:42 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:42.322181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.333903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b204647-955a-459d-a036-c102897aaff0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.347015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.347899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.348320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.348696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.350354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.352223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.369152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.370399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 954/1885] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:42.381711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900cf29f-4a51-4b67-a308-558a7f559cc3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.390374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2460e6a-de65-40d5-885b-25b4c281d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.391072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2460e6a-de65-40d5-885b-25b4c281d4ed admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.391456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2460e6a-de65-40d5-885b-25b4c281d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.391624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2460e6a-de65-40d5-885b-25b4c281d4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.392418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2460e6a-de65-40d5-885b-25b4c281d4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.395340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2460e6a-de65-40d5-885b-25b4c281d4ed admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.408348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 932/1886] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:42.412711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.424025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631360c-de6b-48f0-bedd-43679643dc51 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.434627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.434627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.434627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.434627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] PATH_INFO: `/v3/projects/9fddaafe417448f3b9080fcc02fd6427/users/b014ef6a5a7949f583442d24413ada29/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.447576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fddaafe417448f3b9080fcc02fd6427, user_id=b014ef6a5a7949f583442d24413ada29, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.450298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.465379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 955/1887] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:42 2022] PUT /identity/v3/projects/9fddaafe417448f3b9080fcc02fd6427/users/b014ef6a5a7949f583442d24413ada29/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:42.470511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f4924-a792-4d91-8eac-35028557faca None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.470806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f4924-a792-4d91-8eac-35028557faca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.471087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f4924-a792-4d91-8eac-35028557faca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.491168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d66f4924-a792-4d91-8eac-35028557faca None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:42.492826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d66f4924-a792-4d91-8eac-35028557faca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:42.557697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 933/1888] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:42.567271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:42.569487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:42.637075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3254d67-4d32-4964-b209-3cb22474722c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:42.641633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:42.642590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:42.642954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:42.643366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:42.655414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:42.657015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:42.668361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 956/1889] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:43.622335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:43.622866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:43.623300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:43.650517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] MFA Rules not processed for user `b014ef6a5a7949f583442d24413ada29`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:43.651750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:43.728347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 934/1890] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:43.729071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:43.740338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbccc79-b0b5-4b1b-bf59-f32c6ecb2e02 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:43.757483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:43.758234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:43.758616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:43.758962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:43.759734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:43.761911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:43.795756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 957/1891] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:25:43 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:25:43.800392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:43.812655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe9a14-6c65-435e-bc4f-5b236bbe8f4d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:43.831331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:43.871230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:43.871901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:43.872212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:43.872498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:43.873449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:43.875368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:43.903325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 935/1892] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:25:43 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:25:43.904445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:43.911443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:43.913800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:43.918529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e6cf6-b62d-41e0-9d4b-30d3bff4c0b2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:43.925854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:43.926521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:43.926806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:43.927225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:43.928173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:43.929786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:43.946016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 958/1893] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:25:43.950089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:43.960424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2532a2b-12c2-493a-b60a-0a923f9d6efa admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:43.965496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:43.966030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:43.966313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:43.966657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] PATH_INFO: `/v3/projects/a3c4d2ff3b0b475fb1abf195f37452a0/users/fe5f0640903946329a61f1ba0e5f9fc4/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:43.975816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c7d338-11ec-4c3c-8246-9e3889903b12 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:43.979497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:43.980182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:43.980467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:43.980772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:43.982546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:43.983678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c4d2ff3b0b475fb1abf195f37452a0, user_id=fe5f0640903946329a61f1ba0e5f9fc4, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:43.986583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.004602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 959/1894] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:43 2022] PUT /identity/v3/projects/a3c4d2ff3b0b475fb1abf195f37452a0/users/fe5f0640903946329a61f1ba0e5f9fc4/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:25:44.012687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:44.030861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:44.072058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:44.073565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.084908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 936/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:25:44.133133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92a0fd-b3c7-4bee-807d-47b8e1809221 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:44.154382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9947731a-447d-4676-b524-90b847d800e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:44.155457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947731a-447d-4676-b524-90b847d800e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:44.156040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947731a-447d-4676-b524-90b847d800e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:44.156487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9947731a-447d-4676-b524-90b847d800e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:44.157633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947731a-447d-4676-b524-90b847d800e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:44.159554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9947731a-447d-4676-b524-90b847d800e2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.180509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:44.191286 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 960/1896] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:25:44.194066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd6035-d21b-4170-b5e1-0ad010ea16df service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:44.199427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:44.200602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:44.201046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:44.201793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] PATH_INFO: `/v3/projects/a3c4d2ff3b0b475fb1abf195f37452a0/users/fe5f0640903946329a61f1ba0e5f9fc4/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:44.204862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3c4d2ff3b0b475fb1abf195f37452a0, user_id=fe5f0640903946329a61f1ba0e5f9fc4, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:44.208190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.223331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 937/1897] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:25:44 2022] PUT /identity/v3/projects/a3c4d2ff3b0b475fb1abf195f37452a0/users/fe5f0640903946329a61f1ba0e5f9fc4/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:44.560860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:44.561194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:44.561505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:44.587568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] MFA Rules not processed for user `9f135ce75fbc4b12977cc8f009083ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:44.588366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:44.698745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 961/1898] 10.222.0.26 () {58 vars in 1015 bytes} [Fri Jul 15 19:25:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:44.742547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:44.761796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:44.762264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683efb-87bf-469b-bf3a-01ac95ebd6ea admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:44.763143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:44.787201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81dd3c-0b5f-4d4c-a17c-86be7b353709 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:44.789043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:44.789865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:44.790386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:44.790911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:44.803742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:44.804041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:44.804828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:44.805182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:44.805468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:44.805839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.821336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 938/1899] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:44.822625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:44.836255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:44.838475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4158f9f2-eb16-4a6e-9796-6b64a237cb31 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:44.844012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5af431-2e34-43ec-90e4-004bcb34951a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:44.845195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5af431-2e34-43ec-90e4-004bcb34951a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:44.846008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5af431-2e34-43ec-90e4-004bcb34951a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:44.846429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.846805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5af431-2e34-43ec-90e4-004bcb34951a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:44.870771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 962/1900] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:44.882791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5af431-2e34-43ec-90e4-004bcb34951a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:44.884601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5af431-2e34-43ec-90e4-004bcb34951a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:44.894304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 939/1901] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:44.976384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:45.045465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37be76b-3804-4fea-9e44-4dd9823604f3 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:45.049314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:45.050057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:45.050425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:45.050769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:45.053200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:45.116986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:45.130607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:45.133581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c494815-fbee-44b2-b6ef-da2a21a0b8c6 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:45.146492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 963/1902] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:44 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:46.126922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba8ac-89c9-490c-a934-bf845bf98a67 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:46.127520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba8ac-89c9-490c-a934-bf845bf98a67 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:46.127657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba8ac-89c9-490c-a934-bf845bf98a67 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:46.129226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 940/1903] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:25:46.142974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:46.143825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:46.144133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:46.166783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:46.167795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:46.168212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:46.191041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] MFA Rules not processed for user `fe5f0640903946329a61f1ba0e5f9fc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:46.193660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:46.201509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:46.213981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:46.260057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:46.331466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 964/1904] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:46.347075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:46.364567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3961227e-50b8-4cae-b007-77e23272aee4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:46.375506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a581587-8fda-4491-80aa-e77a9064fe70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:46.376053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a581587-8fda-4491-80aa-e77a9064fe70 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:46.376371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a581587-8fda-4491-80aa-e77a9064fe70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:46.376647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a581587-8fda-4491-80aa-e77a9064fe70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:46.379944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 941/1905] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:25:46.396146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:46.398256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:46.416692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff8a106-a9cf-4d82-b90c-dc6bdc17c71b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:46.426696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324e292-a484-4fc0-8f3f-1c866ad4e2e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:46.428219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324e292-a484-4fc0-8f3f-1c866ad4e2e6 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:46.428864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324e292-a484-4fc0-8f3f-1c866ad4e2e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:46.429217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324e292-a484-4fc0-8f3f-1c866ad4e2e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:46.446023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324e292-a484-4fc0-8f3f-1c866ad4e2e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:46.447720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324e292-a484-4fc0-8f3f-1c866ad4e2e6 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:46.465851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a581587-8fda-4491-80aa-e77a9064fe70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:46.467490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a581587-8fda-4491-80aa-e77a9064fe70 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:46.474866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 942/1906] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:46.482230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 965/1907] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:47.303719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01245f48-9314-4da9-878c-6f7a4b4ff65a None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:47.304433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01245f48-9314-4da9-878c-6f7a4b4ff65a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:47.304529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01245f48-9314-4da9-878c-6f7a4b4ff65a None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:47.306269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 943/1908] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:25:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:25:47.332863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:47.333805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:47.333978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:47.363718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:47.366491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:47.383223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 966/1909] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:47.497678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01245f48-9314-4da9-878c-6f7a4b4ff65a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:47.500147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-01245f48-9314-4da9-878c-6f7a4b4ff65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:47.518233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01245f48-9314-4da9-878c-6f7a4b4ff65a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:47.524632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:47.525279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:47.525567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:47.525846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:47.534529 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:47.536146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:47.546963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 944/1910] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:50.118683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:50.120268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:50.133526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d563728e-225a-4ae8-a3a9-c91630e6f414 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:50.139321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb59a3d-cb76-4861-9238-1aaa07184367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:50.140137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb59a3d-cb76-4861-9238-1aaa07184367 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:50.140500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb59a3d-cb76-4861-9238-1aaa07184367 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:50.140847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb59a3d-cb76-4861-9238-1aaa07184367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:50.151835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb59a3d-cb76-4861-9238-1aaa07184367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:50.153481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb59a3d-cb76-4861-9238-1aaa07184367 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:50.163854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 967/1911] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:57.542813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=31 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:58.961857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:58.964399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:59.055684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e46cc-5f53-4dac-942a-a0b5a5078eff service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:59.060484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:59.064356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:59.064356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:59.064356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] PATH_INFO: `/v3/users/3e514bae272645bab60683f743364911` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:59.064842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e514bae272645bab60683f743364911)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:59.066845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:59.141128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e514bae272645bab60683f743364911'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.149446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e514bae272645bab60683f743364911'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.150139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e514bae272645bab60683f743364911'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.150755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e514bae272645bab60683f743364911'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.158226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e514bae272645bab60683f743364911'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.177956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 945/1912] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:58 2022] DELETE /identity/v3/users/3e514bae272645bab60683f743364911 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:59.182489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:59.183006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:59.183184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:59.202262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:25:59.203611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:59.256098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:25:59.268461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 968/1913] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:25:59.278639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:59.280052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:59.340317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060e636-64cd-496d-a9cd-63382122880a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:59.344396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:59.345050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:59.345333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:59.345608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:59.357297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:59.358794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:59.370135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 946/1914] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:25:59.754045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:59.757441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:25:59.782061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b6980-2844-4e4a-a268-a66c4338c612 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:59.788226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:25:59.789390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:25:59.789743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:25:59.790083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] PATH_INFO: `/v3/projects/9fd48b64c7df4dc4a6a7b9dccf381b99` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:25:59.792235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd48b64c7df4dc4a6a7b9dccf381b99)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:25:59.805868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:25:59.889559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd48b64c7df4dc4a6a7b9dccf381b99', 'request_id': 'req-bb3719c5-0b13-4f08-9f69-841aa576b66c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.900021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd48b64c7df4dc4a6a7b9dccf381b99', 'request_id': 'req-bb3719c5-0b13-4f08-9f69-841aa576b66c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.907206 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd48b64c7df4dc4a6a7b9dccf381b99', 'request_id': 'req-bb3719c5-0b13-4f08-9f69-841aa576b66c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.919178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] The token cache is being invalidate because project 9fd48b64c7df4dc4a6a7b9dccf381b99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:25:59.919641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.920440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:25:59.922029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 969/1915] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:25:59 2022] DELETE /identity/v3/projects/9fd48b64c7df4dc4a6a7b9dccf381b99 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:25:59.926126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:25:59.937966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b9b06a-92b8-4ad4-884e-e524ff3c362f service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:25:59.960712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:00.003344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:00.004255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:00.004587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:00.005068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] PATH_INFO: `/v3/users/0814c9b009c045fc8a87c2af98ae0d06` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:00.006381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0814c9b009c045fc8a87c2af98ae0d06)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:00.008370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:00.056047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0814c9b009c045fc8a87c2af98ae0d06'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.065243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0814c9b009c045fc8a87c2af98ae0d06'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.066445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0814c9b009c045fc8a87c2af98ae0d06'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.067161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0814c9b009c045fc8a87c2af98ae0d06'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.074399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0814c9b009c045fc8a87c2af98ae0d06'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.093425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 947/1916] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:25:59 2022] DELETE /identity/v3/users/0814c9b009c045fc8a87c2af98ae0d06 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:00.458633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:00.460187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:00.474005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3719c5-0b13-4f08-9f69-841aa576b66c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:00.533848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:00.534717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:00.535000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:00.535277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] PATH_INFO: `/v3/projects/3c0d7b13fe5c4e8da17718533e482c01` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:00.537224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0d7b13fe5c4e8da17718533e482c01)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:00.539164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:00.587800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0d7b13fe5c4e8da17718533e482c01', 'request_id': 'req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.597160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0d7b13fe5c4e8da17718533e482c01', 'request_id': 'req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.605080 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0d7b13fe5c4e8da17718533e482c01', 'request_id': 'req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.617053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] The token cache is being invalidate because project 3c0d7b13fe5c4e8da17718533e482c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:00.617531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.618227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.619811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 970/1917] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:00 2022] DELETE /identity/v3/projects/3c0d7b13fe5c4e8da17718533e482c01 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:00.623984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:00.625455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:00.638404 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9ef811-ac87-4cd3-b0ab-169d6b97720f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:00.696706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:00.697616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:00.697912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:00.698176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] PATH_INFO: `/v3/users/69879ba6c71a452caf86ffcceb87a12c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:00.699372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] RBAC: Authorizing `identity:delete_user(user_id=69879ba6c71a452caf86ffcceb87a12c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:00.701443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:00.745622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:00.748707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69879ba6c71a452caf86ffcceb87a12c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.757734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69879ba6c71a452caf86ffcceb87a12c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.758373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69879ba6c71a452caf86ffcceb87a12c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.759430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69879ba6c71a452caf86ffcceb87a12c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.766765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-75bd56e3-9743-402c-82ed-42d27d87085f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69879ba6c71a452caf86ffcceb87a12c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:00.787473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 948/1918] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:00 2022] DELETE /identity/v3/users/69879ba6c71a452caf86ffcceb87a12c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:01.143551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.145737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:01.169603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6a131-a81b-40f6-b450-8e2d6b0bc90d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.175421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:01.233090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.233966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.234308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.234591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] PATH_INFO: `/v3/projects/34ecd63b6a8c4c42b93b2cdd6eb3228b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.235833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] RBAC: Authorizing `identity:delete_project(project_id=34ecd63b6a8c4c42b93b2cdd6eb3228b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.237911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.288140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34ecd63b6a8c4c42b93b2cdd6eb3228b', 'request_id': 'req-18f2a98f-b01a-431a-8640-bc53ab92340e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:01.295164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34ecd63b6a8c4c42b93b2cdd6eb3228b', 'request_id': 'req-18f2a98f-b01a-431a-8640-bc53ab92340e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:01.301323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34ecd63b6a8c4c42b93b2cdd6eb3228b', 'request_id': 'req-18f2a98f-b01a-431a-8640-bc53ab92340e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:01.313315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] The token cache is being invalidate because project 34ecd63b6a8c4c42b93b2cdd6eb3228b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:01.313601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:01.314254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-18f2a98f-b01a-431a-8640-bc53ab92340e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:01.315827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 971/1919] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:01 2022] DELETE /identity/v3/projects/34ecd63b6a8c4c42b93b2cdd6eb3228b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:01.321266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.321710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.321831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.340125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:01.341338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:01.402870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 949/1920] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:01.410999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.411419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.411703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.431812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:01.433054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:01.440999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 972/1921] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:01.445913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.455946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acd0eb-ecf2-45e8-ab35-8ffdf952a8b8 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.460924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.461590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.461880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.462160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.462895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:01.463204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.464856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.478453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 950/1922] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:26:01.482752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.492893 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80834e-d915-4ed8-8d6e-5ca2d9e49328 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.498259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.498922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.499212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.499488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.500216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.502164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.528124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 973/1923] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:01 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:01.531631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.542190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7a2567-b781-4983-84f9-e25dda64e89e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.597633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.598874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.599180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.599461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.600192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.602145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.621614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 951/1924] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:01 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:01.626470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.637086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47100e3a-8123-4e82-beef-819664b4809a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.641616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.642276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.642562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.642842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.643571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.645729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.657034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 974/1925] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:01.661708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.671671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e0de6-c945-4d8b-be8c-95a4cd9e4490 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.676210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c0d88-c77d-4ed3-b691-09c5deaa1a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.677711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c0d88-c77d-4ed3-b691-09c5deaa1a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.678000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c0d88-c77d-4ed3-b691-09c5deaa1a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.678280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c0d88-c77d-4ed3-b691-09c5deaa1a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.679013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c0d88-c77d-4ed3-b691-09c5deaa1a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.680635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c0d88-c77d-4ed3-b691-09c5deaa1a89 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.690890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 952/1926] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:26:01.695166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.705364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e785c-1bc0-40c5-9ca6-b8dd48693c3a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.711284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.712209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.712490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.712764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] PATH_INFO: `/v3/projects/f953904e9d2a4e5a979f0e1788ad9ba8/users/38f01e98de254c42a313885c22859fcb/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.728605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f953904e9d2a4e5a979f0e1788ad9ba8, user_id=38f01e98de254c42a313885c22859fcb, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.731278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.751931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 975/1927] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:01 2022] PUT /identity/v3/projects/f953904e9d2a4e5a979f0e1788ad9ba8/users/38f01e98de254c42a313885c22859fcb/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:01.762447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.762788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.763069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.792303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] MFA Rules not processed for user `38f01e98de254c42a313885c22859fcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:01.793527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:01.807121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:01.865325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 953/1928] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:01.876402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:01.878091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:01.918007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:01.938724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ff9b4-8dcd-4811-bc38-bc37bdc2aad7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:01.943269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c662ab-a072-4264-9734-516bd9fb0905 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:01.943919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c662ab-a072-4264-9734-516bd9fb0905 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:01.944236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c662ab-a072-4264-9734-516bd9fb0905 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:01.944516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c662ab-a072-4264-9734-516bd9fb0905 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:01.953299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c662ab-a072-4264-9734-516bd9fb0905 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:01.954814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c662ab-a072-4264-9734-516bd9fb0905 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:01.964787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 976/1929] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:01.989372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:02.058148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741095f-b1b1-4dfb-8bfd-a29fd1a66875 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:02.062697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:02.063571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:02.063909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:02.064212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] PATH_INFO: `/v3/users/38f01e98de254c42a313885c22859fcb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:02.065418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=38f01e98de254c42a313885c22859fcb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:02.067983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:02.116101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38f01e98de254c42a313885c22859fcb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.123715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38f01e98de254c42a313885c22859fcb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.124441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38f01e98de254c42a313885c22859fcb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.125092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38f01e98de254c42a313885c22859fcb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.133752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38f01e98de254c42a313885c22859fcb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.151917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 954/1930] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:01 2022] DELETE /identity/v3/users/38f01e98de254c42a313885c22859fcb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:02.156445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:02.156738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:02.157019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:02.181451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:02.182623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:02.247531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 977/1931] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:02.258434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:02.260174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:02.305895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:02.325365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7a7e82-2ce3-4d24-ab34-78cef37e2cba admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:02.329129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b4fb6-f867-4177-b8f7-1fef1e3ceace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:02.329857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4fb6-f867-4177-b8f7-1fef1e3ceace service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:02.330217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4fb6-f867-4177-b8f7-1fef1e3ceace service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:02.330568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b4fb6-f867-4177-b8f7-1fef1e3ceace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:02.340781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b4fb6-f867-4177-b8f7-1fef1e3ceace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:02.342270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b4fb6-f867-4177-b8f7-1fef1e3ceace service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:02.353208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 955/1932] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:02.745944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:02.747411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:02.759827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa21ab-33c4-40a3-b9a3-c23c00810fd3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:02.764920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:02.765822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:02.766108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:02.766386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] PATH_INFO: `/v3/projects/f953904e9d2a4e5a979f0e1788ad9ba8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:02.767570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f953904e9d2a4e5a979f0e1788ad9ba8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:02.769632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:02.821779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f953904e9d2a4e5a979f0e1788ad9ba8', 'request_id': 'req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.830028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f953904e9d2a4e5a979f0e1788ad9ba8', 'request_id': 'req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.836533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f953904e9d2a4e5a979f0e1788ad9ba8', 'request_id': 'req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.850369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] The token cache is being invalidate because project f953904e9d2a4e5a979f0e1788ad9ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:02.850655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.851317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4603bfdb-6cad-4d0f-9ab3-2244cb7e89fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:02.852942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 978/1933] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:02 2022] DELETE /identity/v3/projects/f953904e9d2a4e5a979f0e1788ad9ba8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:02.858464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:02.858820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:02.859112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:02.882916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:02.883753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:02.949829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 956/1934] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:02.959181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:02.959496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:02.959782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:02.980477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:02.981858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:02.992933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 979/1935] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:02.997247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.009298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c0bc6a-8410-4a5b-adc3-c1194bb1a7e6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.015341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.016013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.016325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.016704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.017399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:03.017713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.019478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.037834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 957/1936] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:26:03.042189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.052444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68bbbd-4a24-4fa6-aa8e-7f3f024faaa5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.057138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.057792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.058167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.058340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.059024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.060965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.088349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 980/1937] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:03 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:03.092436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.102386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7519afe1-8000-46d7-aeec-572ea7549e45 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.157155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.157829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.158097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.158370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.159052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.161317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.184273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 958/1938] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:03 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:03.185665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.196384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97eb1d1-1b7a-425e-98be-221c8c4bfce9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.201025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.201686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.201964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.202738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.202953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.204759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.218995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 981/1939] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:03.223007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.233727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba24a1-deec-48d3-9f24-d5b428e55a2e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.238337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc978830-9a7e-4866-8289-c9dd1f60a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.239928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc978830-9a7e-4866-8289-c9dd1f60a4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.240255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc978830-9a7e-4866-8289-c9dd1f60a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.240538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc978830-9a7e-4866-8289-c9dd1f60a4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.241272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc978830-9a7e-4866-8289-c9dd1f60a4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.243102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc978830-9a7e-4866-8289-c9dd1f60a4a2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.253704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 959/1940] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:26:03.258454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.268649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e19fab-e13b-4efd-bfd8-814895623b7a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.273948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.274843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.275132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.275412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] PATH_INFO: `/v3/projects/66bf6df84bf2440a9118ef46cb6bd7a5/users/6a457627267044628e156e55873f879e/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.292001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66bf6df84bf2440a9118ef46cb6bd7a5, user_id=6a457627267044628e156e55873f879e, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.294919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.303677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:03.311665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 982/1941] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:03 2022] PUT /identity/v3/projects/66bf6df84bf2440a9118ef46cb6bd7a5/users/6a457627267044628e156e55873f879e/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:03.321216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51605f-84c6-4df8-bbb8-7b08105fb0d9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.322037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51605f-84c6-4df8-bbb8-7b08105fb0d9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.322420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff51605f-84c6-4df8-bbb8-7b08105fb0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.352845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ff51605f-84c6-4df8-bbb8-7b08105fb0d9 None None] MFA Rules not processed for user `6a457627267044628e156e55873f879e`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:03.354159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ff51605f-84c6-4df8-bbb8-7b08105fb0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:03.395328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:03.430729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 960/1942] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:03.441741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:03.443981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:03.502197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39fe86-3a50-4ee3-b007-34bf3bc46367 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:03.506512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14def11a-803c-4b61-8b33-28330c7e9a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:03.507249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14def11a-803c-4b61-8b33-28330c7e9a48 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:03.507442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14def11a-803c-4b61-8b33-28330c7e9a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:03.507719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14def11a-803c-4b61-8b33-28330c7e9a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:03.516250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14def11a-803c-4b61-8b33-28330c7e9a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:03.517734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14def11a-803c-4b61-8b33-28330c7e9a48 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:03.527278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 983/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:09.866310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f07dd1-a3b8-4b5f-bf01-fcc63f703f29 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:09.866783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f07dd1-a3b8-4b5f-bf01-fcc63f703f29 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:09.866953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f07dd1-a3b8-4b5f-bf01-fcc63f703f29 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:09.868907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 961/1944] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:26:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:26:09.876741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:09.877255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:09.877566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:09.901310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:09.913223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:10.001007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 984/1945] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:26:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:10.013900 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f07dd1-a3b8-4b5f-bf01-fcc63f703f29 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:10.015643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-93f07dd1-a3b8-4b5f-bf01-fcc63f703f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:10.033085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f07dd1-a3b8-4b5f-bf01-fcc63f703f29 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:10.038834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941b4b91-1b8b-48e7-88c3-5b6ac28304c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:10.039479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b4b91-1b8b-48e7-88c3-5b6ac28304c3 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:10.039766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b4b91-1b8b-48e7-88c3-5b6ac28304c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:10.040044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941b4b91-1b8b-48e7-88c3-5b6ac28304c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:10.048811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b4b91-1b8b-48e7-88c3-5b6ac28304c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:10.050388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941b4b91-1b8b-48e7-88c3-5b6ac28304c3 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:10.061289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 962/1946] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:12.285631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:12.287582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:12.358798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00145cb4-75c3-4632-96eb-5cfa844e9726 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:12.365255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:12.366150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:12.366435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:12.366712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] PATH_INFO: `/v3/users/4396c6fba1944b22a8291d0713242cd7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:12.367926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4396c6fba1944b22a8291d0713242cd7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:12.369925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:12.406941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:12.417107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4396c6fba1944b22a8291d0713242cd7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:12.424512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4396c6fba1944b22a8291d0713242cd7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:12.425242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4396c6fba1944b22a8291d0713242cd7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:12.425930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4396c6fba1944b22a8291d0713242cd7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:12.438033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4396c6fba1944b22a8291d0713242cd7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:12.449932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 985/1947] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:12 2022] DELETE /identity/v3/users/4396c6fba1944b22a8291d0713242cd7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:12.454436 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:12.454908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:12.455010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:12.472340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:12.474201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:12.532993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 963/1948] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:12.543099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:12.544521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:12.612636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d07d8a-655b-407b-b194-b7576f8ddae6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:12.617392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:12.618045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:12.618324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:12.618601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:12.629228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:12.630779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:12.640436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 986/1949] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:12.986682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:12.989066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:13.004735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c2242-f3ee-4526-b304-f8ed6392f90d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:13.016964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:13.028676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:13.029081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:13.029378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] PATH_INFO: `/v3/projects/cc588be6b3b342af94a6b057523b9708` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:13.034177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc588be6b3b342af94a6b057523b9708)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:13.039538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:13.089790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc588be6b3b342af94a6b057523b9708', 'request_id': 'req-4041be12-8d34-4a37-b7be-a1d42205b839'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.095892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc588be6b3b342af94a6b057523b9708', 'request_id': 'req-4041be12-8d34-4a37-b7be-a1d42205b839'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.106746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc588be6b3b342af94a6b057523b9708', 'request_id': 'req-4041be12-8d34-4a37-b7be-a1d42205b839'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.118731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] The token cache is being invalidate because project cc588be6b3b342af94a6b057523b9708 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:13.119022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.119643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.121226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 964/1950] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:12 2022] DELETE /identity/v3/projects/cc588be6b3b342af94a6b057523b9708 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:13.125601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:13.135848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc8f2b3-7db3-4fa5-b849-d689babc677f service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:13.198111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:13.198993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:13.199278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:13.199558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] PATH_INFO: `/v3/users/0ca2bedfd72646ec9bf6f8683bf2ca0d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:13.200813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca2bedfd72646ec9bf6f8683bf2ca0d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:13.202747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:13.252794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca2bedfd72646ec9bf6f8683bf2ca0d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.261126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca2bedfd72646ec9bf6f8683bf2ca0d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.261776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca2bedfd72646ec9bf6f8683bf2ca0d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.262384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca2bedfd72646ec9bf6f8683bf2ca0d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.265492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:13.279315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca2bedfd72646ec9bf6f8683bf2ca0d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.292621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 987/1951] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:13 2022] DELETE /identity/v3/users/0ca2bedfd72646ec9bf6f8683bf2ca0d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:13.675338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:13.677433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:13.698554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041be12-8d34-4a37-b7be-a1d42205b839 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:13.758581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:13.785952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:13.787370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:13.788646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:13.789083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] PATH_INFO: `/v3/projects/c6a91b7debe64b4baab7f39db033a6dd` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:13.791276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a91b7debe64b4baab7f39db033a6dd)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:13.793290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:13.843792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:13.845322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:13.864362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ede66a9-8eb3-4f2e-8e05-f289b860966a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:13.868783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a91b7debe64b4baab7f39db033a6dd', 'request_id': 'req-39eef431-d032-42c9-95f1-6526b0329da3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.878557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a91b7debe64b4baab7f39db033a6dd', 'request_id': 'req-39eef431-d032-42c9-95f1-6526b0329da3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.885669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a91b7debe64b4baab7f39db033a6dd', 'request_id': 'req-39eef431-d032-42c9-95f1-6526b0329da3'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.903710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] The token cache is being invalidate because project c6a91b7debe64b4baab7f39db033a6dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:13.904274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.904992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-39eef431-d032-42c9-95f1-6526b0329da3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:13.906869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 965/1952] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:13 2022] DELETE /identity/v3/projects/c6a91b7debe64b4baab7f39db033a6dd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:13.912604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7ec1a-8bd5-4f27-96e9-f815b8baf35a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:13.912978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7ec1a-8bd5-4f27-96e9-f815b8baf35a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:13.913333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7ec1a-8bd5-4f27-96e9-f815b8baf35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:13.934653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bae7ec1a-8bd5-4f27-96e9-f815b8baf35a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:13.935958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bae7ec1a-8bd5-4f27-96e9-f815b8baf35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:13.937967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:13.939014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:13.939698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:13.939984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] PATH_INFO: `/v3/users/6a457627267044628e156e55873f879e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:13.941216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a457627267044628e156e55873f879e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:13.943670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:13.945611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 966/1953] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:13.953836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:13.954054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:13.954295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:13.975621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:13.977053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:13.988417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 967/1954] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:13.990192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.000425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdb594-3db3-4ea0-be34-8193d2770142 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.006519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.006881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a457627267044628e156e55873f879e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:14.007199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.007482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.007897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.008838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:14.009148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.010769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.015641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a457627267044628e156e55873f879e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:14.016390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a457627267044628e156e55873f879e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:14.017056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a457627267044628e156e55873f879e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:14.024462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 968/1955] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:26:14.032229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.032493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-45614568-f834-4cda-bc45-7e97bef212f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a457627267044628e156e55873f879e'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:14.034163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:14.047426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 988/1956] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:13 2022] DELETE /identity/v3/users/6a457627267044628e156e55873f879e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:14.050065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1225a3-cb47-4968-97aa-87395123d976 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.051981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.052360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.052715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.081935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:14.084146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:14.117144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.117992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.118350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.118699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.119620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.121652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.158954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:14.161246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 969/1957] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:14 2022] POST /identity/v3/projects => generated 381 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:26:14.166591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.190903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 989/1958] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:14.208562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.212921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:14.219185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:14.251685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51878063-94f6-41eb-9fe0-64fabe374c54 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.258492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.259241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.259626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.259988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.260795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.262802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.283629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 970/1959] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:14 2022] POST /identity/v3/users => generated 291 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:26:14.287846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.297879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615aa08-4eda-4d20-92a1-3a93d21892b0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.302845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b2193-44a6-46bf-af0f-82b6718202a9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.303298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.303904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.304225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.304511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.305242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.306831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.317907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 971/1960] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:26:14.321740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.331536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d03e87-910c-4023-ab90-f85ac70d9382 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.336178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.336836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.337118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.337391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.338104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.339702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.352126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 972/1961] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:26:14.353780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6499327-7b62-476d-80a3-482cce66e6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.354232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.354459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6499327-7b62-476d-80a3-482cce66e6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.354747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6499327-7b62-476d-80a3-482cce66e6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.355027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6499327-7b62-476d-80a3-482cce66e6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.365738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a18c4-08fc-437f-af7a-e20d1b25f4d9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.368473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6499327-7b62-476d-80a3-482cce66e6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.370147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6499327-7b62-476d-80a3-482cce66e6d1 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.376311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.377247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.377532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.377810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] PATH_INFO: `/v3/projects/aabfea9546bf462485922562f91c21f4/users/7ad45814fbff4f52b562f40fb491af7c/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.381165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 990/1962] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:26:14.396968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabfea9546bf462485922562f91c21f4, user_id=7ad45814fbff4f52b562f40fb491af7c, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.399635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.414621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 973/1963] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:14 2022] PUT /identity/v3/projects/aabfea9546bf462485922562f91c21f4/users/7ad45814fbff4f52b562f40fb491af7c/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:14.425073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.425297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.425576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.453962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] MFA Rules not processed for user `7ad45814fbff4f52b562f40fb491af7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:14.455185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:14.545196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 991/1964] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:14.547081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.672383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97712a-eff8-4668-8811-33cefd63ee46 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.685858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.686601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.686805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.687081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.687792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.689951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.729348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 974/1965] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:14 2022] POST /identity/v3/projects => generated 379 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:14.733371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.748368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d40cb62-5ec7-4959-b424-e6af5ebf1e49 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.835977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.835977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:14.836287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:14.861235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.862485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.862958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.863428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.864475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:14.867448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:14.956623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 992/1966] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:14 2022] POST /identity/v3/users => generated 296 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:26:14.961493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:14.984849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab7058-cf94-4684-a03d-e720a6200fb0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:14.996356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:14.997104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:14.997395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:14.997949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:14.998666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:15.000347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:15.025323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef337b61-8328-40c9-9756-40161a8f5035 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:15.027350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 993/1967] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:15.031083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:15.036546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:15.039061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:15.039358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:15.040772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] PATH_INFO: `/v3/projects/66bf6df84bf2440a9118ef46cb6bd7a5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:15.043501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] RBAC: Authorizing `identity:delete_project(project_id=66bf6df84bf2440a9118ef46cb6bd7a5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:15.049713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:15.055865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f3fb1-c639-4188-8d99-44a8a4f14e29 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:15.057986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:15.067719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:15.068759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:15.069047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:15.069329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] PATH_INFO: `/v3/projects/693aeb79ae49442d998dac5162b91706/users/4a32d45e25ca4e41a67b0dc6378e0282/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:15.106367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=693aeb79ae49442d998dac5162b91706, user_id=4a32d45e25ca4e41a67b0dc6378e0282, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:15.109123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:15.133578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 994/1968] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:15 2022] PUT /identity/v3/projects/693aeb79ae49442d998dac5162b91706/users/4a32d45e25ca4e41a67b0dc6378e0282/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:15.137204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:15.161700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66bf6df84bf2440a9118ef46cb6bd7a5', 'request_id': 'req-8c05424b-ecf9-42e0-b389-35a84d43beec'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:15.173493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66bf6df84bf2440a9118ef46cb6bd7a5', 'request_id': 'req-8c05424b-ecf9-42e0-b389-35a84d43beec'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:15.192807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66bf6df84bf2440a9118ef46cb6bd7a5', 'request_id': 'req-8c05424b-ecf9-42e0-b389-35a84d43beec'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:15.216371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] The token cache is being invalidate because project 66bf6df84bf2440a9118ef46cb6bd7a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:15.218561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:15.219764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:15.222821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 975/1969] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:14 2022] DELETE /identity/v3/projects/66bf6df84bf2440a9118ef46cb6bd7a5 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:15.262805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d34228-1348-4e8e-9de3-b434c5cc6d64 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:15.377372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5eae0d-5e4b-47ee-8136-997a03f4bf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:15.379065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eae0d-5e4b-47ee-8136-997a03f4bf5d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:15.379640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eae0d-5e4b-47ee-8136-997a03f4bf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:15.380209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5eae0d-5e4b-47ee-8136-997a03f4bf5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:15.381374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5eae0d-5e4b-47ee-8136-997a03f4bf5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:15.383341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5eae0d-5e4b-47ee-8136-997a03f4bf5d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:15.397191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 995/1970] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:15.401173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:15.403691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:15.421984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05424b-ecf9-42e0-b389-35a84d43beec admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:15.430265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:15.431748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:15.432296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:15.432588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] PATH_INFO: `/v3/projects/693aeb79ae49442d998dac5162b91706/users/4a32d45e25ca4e41a67b0dc6378e0282/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:15.434762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=693aeb79ae49442d998dac5162b91706, user_id=4a32d45e25ca4e41a67b0dc6378e0282, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:15.438091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:15.452255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 976/1971] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:15 2022] PUT /identity/v3/projects/693aeb79ae49442d998dac5162b91706/users/4a32d45e25ca4e41a67b0dc6378e0282/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:15.462660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:15.462947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:15.463710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:15.500364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] MFA Rules not processed for user `4a32d45e25ca4e41a67b0dc6378e0282`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:15.501598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:15.534960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:15.587229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 996/1972] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:15.603007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:15.605209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:15.689889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b9c8b-de5d-4019-b52b-2246e177acbc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:15.696347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:15.697063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:15.697483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:15.697893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:15.712764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:15.714320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:15.742585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 977/1973] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:16.228616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:16.239961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae455d-50e9-4c03-8bca-554e91d45021 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:16.244640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32d45e25ca4e41a67b0dc6378e0282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693aeb79ae49442d998dac5162b91706', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:16.245312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:16.245607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:16.245885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:16.246628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:16.246938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:16.249028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:16.266592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:16.272488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 997/1974] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:26:16.279861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919885f7-1f47-48f9-8c87-3eb73f77e59f service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:16.288295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32d45e25ca4e41a67b0dc6378e0282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693aeb79ae49442d998dac5162b91706', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:16.288295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:16.288295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:16.288551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:16.288551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:16.290225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:16.328385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 978/1975] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:16 2022] POST /identity/v3/projects => generated 353 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:26:16.335153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:16.349422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35565b0-b119-4e35-a950-a34e6fe5e912 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:16.416514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32d45e25ca4e41a67b0dc6378e0282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693aeb79ae49442d998dac5162b91706', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:16.417102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:16.417422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:16.417707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:16.418514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:16.418856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:16.420494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:16.435845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 998/1976] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:26:16.471076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:16.483245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6ce24-ccd7-43a1-8b7f-0424c7c7d8fa tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:16.487732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a32d45e25ca4e41a67b0dc6378e0282', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693aeb79ae49442d998dac5162b91706', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:16.489308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:16.489593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:16.489873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] PATH_INFO: `/v3/projects/b449b1f491e44f10a77489d2c5ebf3df` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:16.491078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b449b1f491e44f10a77489d2c5ebf3df)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:16.494782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:16.549042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b449b1f491e44f10a77489d2c5ebf3df', 'request_id': 'req-792120a3-5d1b-4cb8-97ce-ba7328ed420d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:16.557395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b449b1f491e44f10a77489d2c5ebf3df', 'request_id': 'req-792120a3-5d1b-4cb8-97ce-ba7328ed420d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:16.558750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:16.566070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b449b1f491e44f10a77489d2c5ebf3df', 'request_id': 'req-792120a3-5d1b-4cb8-97ce-ba7328ed420d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:16.583580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] The token cache is being invalidate because project b449b1f491e44f10a77489d2c5ebf3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:16.584247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:16.585793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-792120a3-5d1b-4cb8-97ce-ba7328ed420d tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:16.587399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 979/1977] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:16 2022] DELETE /identity/v3/projects/b449b1f491e44f10a77489d2c5ebf3df => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:16.799085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:16.800831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:16.876679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b0cb7-4ad5-4229-8688-013d20683b10 tempest-VolumeQuotaClassesTest-779898161 tempest-VolumeQuotaClassesTest-779898161-project-admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:16.939854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:16.951670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:16.952713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:16.952897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:16.953142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] PATH_INFO: `/v3/users/7ad45814fbff4f52b562f40fb491af7c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:16.954367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad45814fbff4f52b562f40fb491af7c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:16.956359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:17.008842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad45814fbff4f52b562f40fb491af7c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.016257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad45814fbff4f52b562f40fb491af7c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.016901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad45814fbff4f52b562f40fb491af7c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.017511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad45814fbff4f52b562f40fb491af7c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.032155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad45814fbff4f52b562f40fb491af7c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.044856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 999/1978] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:16 2022] DELETE /identity/v3/users/7ad45814fbff4f52b562f40fb491af7c => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:17.049398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7718-e350-4428-bef8-799361e0202e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:17.049862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7718-e350-4428-bef8-799361e0202e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:17.050096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e7718-e350-4428-bef8-799361e0202e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:17.068583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-695e7718-e350-4428-bef8-799361e0202e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:17.069931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-695e7718-e350-4428-bef8-799361e0202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:17.132037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 980/1979] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:17.143000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:17.144841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:17.216268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2e3d2-a26c-4443-aa2d-d93bbf280e00 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:17.221901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:17.222564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:17.222847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:17.223158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:17.233504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:17.234987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:17.245974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1000/1980] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:17.511431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7718-e350-4428-bef8-799361e0202e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:17.512972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-695e7718-e350-4428-bef8-799361e0202e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:17.531497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e7718-e350-4428-bef8-799361e0202e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:17.538083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:17.539003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:17.539288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:17.539584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] PATH_INFO: `/v3/projects/aabfea9546bf462485922562f91c21f4` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:17.540849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabfea9546bf462485922562f91c21f4)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:17.542824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:17.588783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabfea9546bf462485922562f91c21f4', 'request_id': 'req-8101d941-54e5-436e-9ec5-affb1a8a6252'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.594803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabfea9546bf462485922562f91c21f4', 'request_id': 'req-8101d941-54e5-436e-9ec5-affb1a8a6252'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.603855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabfea9546bf462485922562f91c21f4', 'request_id': 'req-8101d941-54e5-436e-9ec5-affb1a8a6252'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.617314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] The token cache is being invalidate because project aabfea9546bf462485922562f91c21f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:17.617606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.618225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.619808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 981/1981] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:17 2022] DELETE /identity/v3/projects/aabfea9546bf462485922562f91c21f4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:17.624024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:17.635048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072028c9-b6de-4907-853c-84ca4005ea89 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:17.692817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:17.693815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:17.694104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:17.694382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] PATH_INFO: `/v3/users/4a32d45e25ca4e41a67b0dc6378e0282` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:17.695586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a32d45e25ca4e41a67b0dc6378e0282)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:17.697562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:17.715506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:17.756434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a32d45e25ca4e41a67b0dc6378e0282'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.775963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a32d45e25ca4e41a67b0dc6378e0282'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.776622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a32d45e25ca4e41a67b0dc6378e0282'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.777323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a32d45e25ca4e41a67b0dc6378e0282'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.795299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-650b639f-6522-427f-abab-4d9c1f4c25bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a32d45e25ca4e41a67b0dc6378e0282'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:17.818925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1001/1982] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:17 2022] DELETE /identity/v3/users/4a32d45e25ca4e41a67b0dc6378e0282 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:18.328350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.330756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:18.345954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101d941-54e5-436e-9ec5-affb1a8a6252 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.436549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.437432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.437868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.438153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] PATH_INFO: `/v3/projects/693aeb79ae49442d998dac5162b91706` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.439396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=693aeb79ae49442d998dac5162b91706)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:18.441402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:18.451545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:18.496503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693aeb79ae49442d998dac5162b91706', 'request_id': 'req-0eaf0d07-f311-486a-9073-d464844b8f98'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:18.503605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693aeb79ae49442d998dac5162b91706', 'request_id': 'req-0eaf0d07-f311-486a-9073-d464844b8f98'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:18.509298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693aeb79ae49442d998dac5162b91706', 'request_id': 'req-0eaf0d07-f311-486a-9073-d464844b8f98'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:18.521253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] The token cache is being invalidate because project 693aeb79ae49442d998dac5162b91706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:18.521540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:18.522159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0eaf0d07-f311-486a-9073-d464844b8f98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:18.523703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 982/1983] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:18 2022] DELETE /identity/v3/projects/693aeb79ae49442d998dac5162b91706 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:18.529496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.529788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.530064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.548452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:18.549630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:18.608652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1002/1984] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:18.618147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.618439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.618718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.637342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:18.638644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:18.646735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 983/1985] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:18.651049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.662594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d8138-989e-4eb9-926a-c0e3393c5b00 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.669103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.669262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.669532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.669809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.670549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:18.670857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:18.672504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:18.690047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1003/1986] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:26:18.691325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.700625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80030058-5ba6-437e-8f07-5cc2b8577357 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.705859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc21b418-c124-4224-8e46-950154ef0770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.706516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21b418-c124-4224-8e46-950154ef0770 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.706797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21b418-c124-4224-8e46-950154ef0770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.707073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc21b418-c124-4224-8e46-950154ef0770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.708565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21b418-c124-4224-8e46-950154ef0770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:18.710478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc21b418-c124-4224-8e46-950154ef0770 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:18.749953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 984/1987] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:18 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:18.751120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.761583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da528379-582b-43db-b0a1-5b72525d67c4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.774022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d046b-8659-4edd-b4f7-7f89d0f6a0dd None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.774652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d046b-8659-4edd-b4f7-7f89d0f6a0dd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.775481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d046b-8659-4edd-b4f7-7f89d0f6a0dd None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.777090 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 985/1988] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:26:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:26:18.813702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.813985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.814264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.820494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.821471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.821753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.822840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.823591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:18.825538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:18.837479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:18.838731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:18.851277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1004/1989] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:18 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:18.872816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.899220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f57bc-ac63-43af-a807-c422dd04139e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.905313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.906012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.906296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.906572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.907307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:18.909408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:18.917204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:18.924569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1005/1990] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:18.929324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.944984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf9f25-a824-4e05-af16-57af5f576259 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.948982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 986/1991] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:26:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:18.950550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.951468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.951955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.953594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:18.953594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:18.955609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:18.970081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1006/1992] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:18.974935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:18.985743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee34b2-aae3-4a36-bc97-19abc25873ec None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:18.991014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9401421d-79bd-431e-ad1d-b10e0828f7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:18.992236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401421d-79bd-431e-ad1d-b10e0828f7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:18.992608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401421d-79bd-431e-ad1d-b10e0828f7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:18.992892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9401421d-79bd-431e-ad1d-b10e0828f7d6 admin admin] PATH_INFO: `/v3/projects/7998a53aaca0417aa89d905f03b923c6/users/918edf180b5b40f58994469fb37d25c2/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:19.013464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401421d-79bd-431e-ad1d-b10e0828f7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7998a53aaca0417aa89d905f03b923c6, user_id=918edf180b5b40f58994469fb37d25c2, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:19.018085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9401421d-79bd-431e-ad1d-b10e0828f7d6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:19.028963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:19.032516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:19.054150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efb119d-dc54-47b2-a6f0-9cd52452efc4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:19.062727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 987/1993] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:18 2022] PUT /identity/v3/projects/7998a53aaca0417aa89d905f03b923c6/users/918edf180b5b40f58994469fb37d25c2/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:19.068231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:19.068530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:19.068811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:19.094067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:19.095338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:19.140586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:19.142413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67c1d08-9060-4488-8144-8d8bf6216b7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:19.143332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c1d08-9060-4488-8144-8d8bf6216b7a service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:19.143826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c1d08-9060-4488-8144-8d8bf6216b7a service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:19.144417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67c1d08-9060-4488-8144-8d8bf6216b7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:19.156466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c1d08-9060-4488-8144-8d8bf6216b7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:19.158860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67c1d08-9060-4488-8144-8d8bf6216b7a service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:19.179793 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1007/1994] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:19.183842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 988/1995] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:26:19.196137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:19.197823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:19.270294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e29fa-dff7-49b1-8155-581e75b5937f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:19.274895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237e0f4-8b5e-4b0e-b99f-d9371a1fc882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:19.276536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237e0f4-8b5e-4b0e-b99f-d9371a1fc882 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:19.277189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237e0f4-8b5e-4b0e-b99f-d9371a1fc882 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:19.277633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237e0f4-8b5e-4b0e-b99f-d9371a1fc882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:19.307857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237e0f4-8b5e-4b0e-b99f-d9371a1fc882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:19.309468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237e0f4-8b5e-4b0e-b99f-d9371a1fc882 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:19.321187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 989/1996] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:19.725631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d81d2e-a061-4b43-af90-800c7bfcbbbe None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:19.725631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d81d2e-a061-4b43-af90-800c7bfcbbbe None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:19.726172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d81d2e-a061-4b43-af90-800c7bfcbbbe None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:19.731159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1008/1997] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:26:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:26:19.742616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:19.743127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:19.743215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:19.776983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:19.780819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:19.797640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 990/1998] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:26:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:19.992818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d81d2e-a061-4b43-af90-800c7bfcbbbe None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:20.009258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d81d2e-a061-4b43-af90-800c7bfcbbbe None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:20.015843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666a8e0-416f-4b37-a7c2-8a0b9efdd241 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:20.016941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666a8e0-416f-4b37-a7c2-8a0b9efdd241 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:20.017527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666a8e0-416f-4b37-a7c2-8a0b9efdd241 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:20.018012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666a8e0-416f-4b37-a7c2-8a0b9efdd241 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:20.029641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666a8e0-416f-4b37-a7c2-8a0b9efdd241 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:20.031407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666a8e0-416f-4b37-a7c2-8a0b9efdd241 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:20.043688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1009/1999] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:20.056006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:20.058480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:20.085962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67c782-a3c3-4e09-8908-40ed28575351 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:20.094898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:20.096849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:20.096849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:20.096849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] PATH_INFO: `/v3/users/b014ef6a5a7949f583442d24413ada29` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:20.098813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b014ef6a5a7949f583442d24413ada29)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:20.102485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:20.174095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b014ef6a5a7949f583442d24413ada29'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.183247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b014ef6a5a7949f583442d24413ada29'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.183967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b014ef6a5a7949f583442d24413ada29'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.184633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b014ef6a5a7949f583442d24413ada29'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.193084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b014ef6a5a7949f583442d24413ada29'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.211440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 991/2000] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:20 2022] DELETE /identity/v3/users/b014ef6a5a7949f583442d24413ada29 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:20.429354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:20.429645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:20.429927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:20.465437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] MFA Rules not processed for user `918edf180b5b40f58994469fb37d25c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:20.466831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:20.493832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:20.500145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:20.577303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1010/2001] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:20.581594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:20.589897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:20.595799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:20.609850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f026a4c4-1564-46a8-b416-f5c695fad09d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:20.616224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:20.617276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:20.617858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:20.618312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] PATH_INFO: `/v3/projects/9fddaafe417448f3b9080fcc02fd6427` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:20.619712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fddaafe417448f3b9080fcc02fd6427)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:20.622211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:20.669218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480954e-6dd1-4c91-b3b9-38ca2d63668f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:20.674130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:20.675055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:20.675726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:20.676839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:20.687430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:20.689106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:20.693042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:20.700013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1011/2002] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:20.712330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fddaafe417448f3b9080fcc02fd6427', 'request_id': 'req-5edec4f7-baa3-47ed-af7f-6720570f8374'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.719039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fddaafe417448f3b9080fcc02fd6427', 'request_id': 'req-5edec4f7-baa3-47ed-af7f-6720570f8374'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.726414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fddaafe417448f3b9080fcc02fd6427', 'request_id': 'req-5edec4f7-baa3-47ed-af7f-6720570f8374'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.747824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] The token cache is being invalidate because project 9fddaafe417448f3b9080fcc02fd6427 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:20.748418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.748878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.750706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 992/2003] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:20 2022] DELETE /identity/v3/projects/9fddaafe417448f3b9080fcc02fd6427 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:20.755318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:20.782360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616c6e9-e99c-4ab5-824a-101c04378ac7 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:20.871107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:20.872242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:20.872924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:20.873205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] PATH_INFO: `/v3/users/fe5f0640903946329a61f1ba0e5f9fc4` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:20.874766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5f0640903946329a61f1ba0e5f9fc4)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:20.877239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:20.965004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5f0640903946329a61f1ba0e5f9fc4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.975689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5f0640903946329a61f1ba0e5f9fc4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.977117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5f0640903946329a61f1ba0e5f9fc4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.977740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5f0640903946329a61f1ba0e5f9fc4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:20.995223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c10f7fa1-1f33-48b2-bb89-e854a58d9bee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5f0640903946329a61f1ba0e5f9fc4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:21.014354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1012/2004] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:20 2022] DELETE /identity/v3/users/fe5f0640903946329a61f1ba0e5f9fc4 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:21.217403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.218994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:21.237745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edec4f7-baa3-47ed-af7f-6720570f8374 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.296163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.297028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.297307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.297579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] PATH_INFO: `/v3/projects/a3c4d2ff3b0b475fb1abf195f37452a0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.298770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3c4d2ff3b0b475fb1abf195f37452a0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.300733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.349924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3c4d2ff3b0b475fb1abf195f37452a0', 'request_id': 'req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:21.356240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3c4d2ff3b0b475fb1abf195f37452a0', 'request_id': 'req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:21.362001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3c4d2ff3b0b475fb1abf195f37452a0', 'request_id': 'req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:21.377581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] The token cache is being invalidate because project a3c4d2ff3b0b475fb1abf195f37452a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:21.378140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:21.378778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6f3cfe6d-f833-4a14-aefa-ad8d7b075fb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:21.380955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 993/2005] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:21 2022] DELETE /identity/v3/projects/a3c4d2ff3b0b475fb1abf195f37452a0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:21.385750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.386034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.386324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.404536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:21.405758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:21.467954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1013/2006] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:21.476511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.476808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.477127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.497134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:21.499090 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:21.508986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 994/2007] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:21.513463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.523221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85c40f-2894-40f1-8411-8af8aeed8514 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.527815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.528504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.528786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.529061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.529787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:21.530090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.531692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.543990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1014/2008] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:26:21.548624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.553820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:21.560168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21bfa02-f8a4-40ac-ae47-e963eb607dae None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.566351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.568229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.568556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.568844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.569546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.571470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.598856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 995/2009] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:21 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:26:21.602983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.609522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:21.613276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5f7a4-2829-4dab-8f9d-022970a4257e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.680187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.680868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.681152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.681430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.682150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.684378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.706249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1015/2010] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:21 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:26:21.710376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.721378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24498751-5b96-47be-b497-7e56d51b4593 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.727102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.727802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.728115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.728400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.729141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.730750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.743801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 996/2011] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:21.748263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.759759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ca6b6e-5164-42ad-8065-7133ca66345f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.764632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c7ca4-b442-4926-814c-383d0b758291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.765374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c7ca4-b442-4926-814c-383d0b758291 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.765655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c7ca4-b442-4926-814c-383d0b758291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.765932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c7ca4-b442-4926-814c-383d0b758291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.766693 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c7ca4-b442-4926-814c-383d0b758291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.768479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c7ca4-b442-4926-814c-383d0b758291 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.780450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1016/2012] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:26:21.784645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.795005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc3f07-02cc-44c5-a6b7-45ab66d40e57 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:21.799557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:21.800472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.800750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.801559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] PATH_INFO: `/v3/projects/7400e4952ed643d0a72d7de178bdacd3/users/6a934fbe7ba9404ca2d1f36919961cc9/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.818312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7400e4952ed643d0a72d7de178bdacd3, user_id=6a934fbe7ba9404ca2d1f36919961cc9, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:21.821274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:21.835886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 997/2013] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:21 2022] PUT /identity/v3/projects/7400e4952ed643d0a72d7de178bdacd3/users/6a934fbe7ba9404ca2d1f36919961cc9/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:21.847063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:21.847358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:21.847641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:21.875448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] MFA Rules not processed for user `6a934fbe7ba9404ca2d1f36919961cc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:21.876681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:21.940746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1017/2014] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:21.952730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:21.954424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:22.017391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4a201-8d72-4a25-9890-665785b4e7d2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:22.022487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7377d-6860-4b74-9b5e-69fecd77a69b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:22.023192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7377d-6860-4b74-9b5e-69fecd77a69b service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:22.023562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7377d-6860-4b74-9b5e-69fecd77a69b service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:22.023986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7377d-6860-4b74-9b5e-69fecd77a69b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:22.033794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7377d-6860-4b74-9b5e-69fecd77a69b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:22.035258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7377d-6860-4b74-9b5e-69fecd77a69b service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:22.044860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 998/2015] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:22.065706 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:22.177806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82b098-97ae-4b43-8d88-faa176bb1fd4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:22.178206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82b098-97ae-4b43-8d88-faa176bb1fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:22.178489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82b098-97ae-4b43-8d88-faa176bb1fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:22.193226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7f563-1059-4025-8dfb-5873268b143a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:22.209195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-da82b098-97ae-4b43-8d88-faa176bb1fd4 None None] MFA Rules not processed for user `918edf180b5b40f58994469fb37d25c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:22.211033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-da82b098-97ae-4b43-8d88-faa176bb1fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:22.211157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:22.212056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:22.212375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:22.212655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] PATH_INFO: `/v3/users/6a934fbe7ba9404ca2d1f36919961cc9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:22.217117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a934fbe7ba9404ca2d1f36919961cc9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:22.219847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:22.237290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:22.308234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 999/2016] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:22.312023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a934fbe7ba9404ca2d1f36919961cc9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:22.326005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a934fbe7ba9404ca2d1f36919961cc9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:22.326735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a934fbe7ba9404ca2d1f36919961cc9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:22.327633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a934fbe7ba9404ca2d1f36919961cc9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:22.344996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a934fbe7ba9404ca2d1f36919961cc9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:22.364172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1018/2017] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:22 2022] DELETE /identity/v3/users/6a934fbe7ba9404ca2d1f36919961cc9 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:22.365603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:22.366231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:22.367772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:22.403426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:22.405046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:22.474979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:22.498432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1000/2018] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:22.538228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:22.540161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:22.673487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0dcb57-10ea-4b0c-8f87-3cf30b26055e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:22.678243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babe1ce-cc07-482c-aeb8-ba250ecf9d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:22.679249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe1ce-cc07-482c-aeb8-ba250ecf9d7f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:22.679644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe1ce-cc07-482c-aeb8-ba250ecf9d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:22.680128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babe1ce-cc07-482c-aeb8-ba250ecf9d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:22.692248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babe1ce-cc07-482c-aeb8-ba250ecf9d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:22.693697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1babe1ce-cc07-482c-aeb8-ba250ecf9d7f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:22.704723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1019/2019] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:23.181291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.182771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:23.198252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4dd0-7b53-48e4-8f3c-e06dab63a0d0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.204382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.205600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.205955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.206176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] PATH_INFO: `/v3/projects/7400e4952ed643d0a72d7de178bdacd3` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.207411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7400e4952ed643d0a72d7de178bdacd3)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.210799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.271994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7400e4952ed643d0a72d7de178bdacd3', 'request_id': 'req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:23.279235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7400e4952ed643d0a72d7de178bdacd3', 'request_id': 'req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:23.285506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7400e4952ed643d0a72d7de178bdacd3', 'request_id': 'req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:23.301678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] The token cache is being invalidate because project 7400e4952ed643d0a72d7de178bdacd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:23.302237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:23.303394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a6b37c8-5881-4d73-b182-d83f28c7f7d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:23.308359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1001/2020] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:23 2022] DELETE /identity/v3/projects/7400e4952ed643d0a72d7de178bdacd3 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:23.310571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.311011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.311110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.336889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:23.338163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:23.425517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1020/2021] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:23.427167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.427461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.427743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.448605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:23.450055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:23.460013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1002/2022] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:23.464379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.474468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1709dbef-c551-4fcd-b471-f1a0f252a918 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.480312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.480988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.481271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.481547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.482280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:26:23.483085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.484765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.516379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1021/2023] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:26:23.517573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.528739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab6f83-dc8c-4c24-9ad7-781fb5f6bb3b None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.533650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.534447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.534797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.535143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.536370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.538396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.570920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1003/2024] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:26:23 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:23.575028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.585505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce2ccc-307b-4927-bc94-d5fb3cdbb755 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.614492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:23.659329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.662342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.662342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.662342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.663245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.665313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.705661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1022/2025] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:26:23 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:26:23.708590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.729963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa9ca2-edef-41b7-88b5-1b9605f8197b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.737451 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.739201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.739898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.740370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.742366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.744619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.770370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1004/2026] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:26:23.772586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.782447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f923e2-d4f9-4a41-9175-ad69f8336ab1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.788320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed49d4-415e-4cdc-819e-caf7db146fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.789260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed49d4-415e-4cdc-819e-caf7db146fda admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.789780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed49d4-415e-4cdc-819e-caf7db146fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.790262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed49d4-415e-4cdc-819e-caf7db146fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.791252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed49d4-415e-4cdc-819e-caf7db146fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.793330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed49d4-415e-4cdc-819e-caf7db146fda admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.804976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1023/2027] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:26:23.809241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.820137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4857cee-b6d4-4890-9042-7b6be174d9ec admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:23.824774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:23.825781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.826137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.826482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] PATH_INFO: `/v3/projects/5b5ff7269cb24d8f802e21cdea271404/users/40dc78f21e78469c8b10d3e3fdea08d9/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.843838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5ff7269cb24d8f802e21cdea271404, user_id=40dc78f21e78469c8b10d3e3fdea08d9, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:23.846686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:23.861188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1005/2028] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:26:23 2022] PUT /identity/v3/projects/5b5ff7269cb24d8f802e21cdea271404/users/40dc78f21e78469c8b10d3e3fdea08d9/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:23.871677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:23.871972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:23.872279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:23.905947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] MFA Rules not processed for user `40dc78f21e78469c8b10d3e3fdea08d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:23.907260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:23.983082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1024/2029] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:23.995310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:23.997246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:24.069369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443f132-9d21-4e0b-a0bb-799265661c83 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:24.078626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3b5d2-4815-4b3a-94ee-8bb71be6a315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:24.079308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b5d2-4815-4b3a-94ee-8bb71be6a315 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:24.079593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b5d2-4815-4b3a-94ee-8bb71be6a315 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:24.079871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3b5d2-4815-4b3a-94ee-8bb71be6a315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:24.089631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3b5d2-4815-4b3a-94ee-8bb71be6a315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:24.091292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3b5d2-4815-4b3a-94ee-8bb71be6a315 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:24.103396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1006/2030] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:33.979996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:34.097965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:26:49.318725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:49.380106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96441f6c-70c1-41e8-8b20-7c13648cba71 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:49.384517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:49.385383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:49.385665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:49.385942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] PATH_INFO: `/v3/users/40dc78f21e78469c8b10d3e3fdea08d9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:49.387123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40dc78f21e78469c8b10d3e3fdea08d9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:49.389102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:49.434320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40dc78f21e78469c8b10d3e3fdea08d9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:49.441904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40dc78f21e78469c8b10d3e3fdea08d9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:49.442500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40dc78f21e78469c8b10d3e3fdea08d9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:49.443093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40dc78f21e78469c8b10d3e3fdea08d9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:49.454595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40dc78f21e78469c8b10d3e3fdea08d9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:49.469094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1025/2031] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:26:49 2022] DELETE /identity/v3/users/40dc78f21e78469c8b10d3e3fdea08d9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:26:49.474271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:49.474795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:49.474888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:49.492337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:26:49.493548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:49.553323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1007/2032] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:26:49.563086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:49.564539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:49.622988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799bbfd-90f2-47e6-97b7-d2c686527ea1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:49.626665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:49.627307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:49.627591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:49.627872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:49.637685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:49.639062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:49.648516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1026/2033] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:26:49.919147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:26:49.920693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:26:49.933670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432751-cf6a-4628-bb37-4591fe10bd67 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:26:49.938987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:26:49.939899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:26:49.940237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:26:49.940535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] PATH_INFO: `/v3/projects/5b5ff7269cb24d8f802e21cdea271404` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:26:49.948661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5ff7269cb24d8f802e21cdea271404)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:26:49.950653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:26:49.998075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5ff7269cb24d8f802e21cdea271404', 'request_id': 'req-752a362f-0f81-47c7-a019-592094e1616c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:50.005790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5ff7269cb24d8f802e21cdea271404', 'request_id': 'req-752a362f-0f81-47c7-a019-592094e1616c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:50.011720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5ff7269cb24d8f802e21cdea271404', 'request_id': 'req-752a362f-0f81-47c7-a019-592094e1616c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:50.024114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] The token cache is being invalidate because project 5b5ff7269cb24d8f802e21cdea271404 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:26:50.024297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:50.024962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:26:50.026526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1008/2034] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:26:49 2022] DELETE /identity/v3/projects/5b5ff7269cb24d8f802e21cdea271404 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:26:59.644374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:00.020968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:06.419934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:06.421529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:06.437453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690215ce-13e3-4882-a80d-0202cfff6bdb service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:06.497900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:06.498862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:06.499167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:06.499491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] PATH_INFO: `/v3/users/d06d1b5c0eff4645bf7fb43f8921bb17` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:06.500766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d06d1b5c0eff4645bf7fb43f8921bb17)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:06.502724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:06.556518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd06d1b5c0eff4645bf7fb43f8921bb17'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.563762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd06d1b5c0eff4645bf7fb43f8921bb17'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.564394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd06d1b5c0eff4645bf7fb43f8921bb17'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.565010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd06d1b5c0eff4645bf7fb43f8921bb17'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.584533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ec61bf9b-ec14-4602-a6ba-8c2d16a5ed04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd06d1b5c0eff4645bf7fb43f8921bb17'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.598465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1027/2035] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:06 2022] DELETE /identity/v3/users/d06d1b5c0eff4645bf7fb43f8921bb17 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:06.771477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:06.773007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:06.786304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a362f-0f81-47c7-a019-592094e1616c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:06.844888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:06.845779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:06.846069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:06.846343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] PATH_INFO: `/v3/projects/b2599f1a3d5d40a4add00e3343d1c4b4` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:06.847613 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2599f1a3d5d40a4add00e3343d1c4b4)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:06.849771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:06.897503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2599f1a3d5d40a4add00e3343d1c4b4', 'request_id': 'req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.903696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2599f1a3d5d40a4add00e3343d1c4b4', 'request_id': 'req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.909240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2599f1a3d5d40a4add00e3343d1c4b4', 'request_id': 'req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.921569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] The token cache is being invalidate because project b2599f1a3d5d40a4add00e3343d1c4b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:06.921858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.922477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d6bdddeb-b0e4-4d7c-843c-9af8b37531d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:06.924016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1009/2036] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:06 2022] DELETE /identity/v3/projects/b2599f1a3d5d40a4add00e3343d1c4b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:06.929501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:06.929794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:06.930074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:06.948686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:06.950044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:07.015614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1028/2037] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:07.024042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.024369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.024653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.042614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:07.043819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:07.051663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1010/2038] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:07.055823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.065970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6439015-ecff-4972-9dc3-91217587b825 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.071614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.072329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.072613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.072890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.073630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:07.073943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.075560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.092050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1029/2039] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:27:07.093136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.102310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7483c3-47fa-42a7-812c-362e026f4f58 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.106846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.107498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.107802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.108061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.108799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.110700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.146929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1011/2040] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:07 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:07.148301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.158378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1de4d-9d8c-46ea-8762-f426000b3589 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.162463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:07.227675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.229124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.229417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.229831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.230672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.232721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.264863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1030/2041] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:07 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:27:07.266039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.276305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdfbbd-6f76-4bd8-bac6-69396e082acf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.283429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.284131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.284440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.284747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.286321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.287961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.299682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1012/2042] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:07.304441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.314827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147cb0-b09b-4a6b-bfbb-e72a0bbd410a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.319610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbc2684-f7e4-4163-9fdc-158e9dacf0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.320301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc2684-f7e4-4163-9fdc-158e9dacf0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.320581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc2684-f7e4-4163-9fdc-158e9dacf0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.320857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbc2684-f7e4-4163-9fdc-158e9dacf0c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.321573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc2684-f7e4-4163-9fdc-158e9dacf0c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.323168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbc2684-f7e4-4163-9fdc-158e9dacf0c7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.333066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1031/2043] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:07.337111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.349560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8dae-7ca7-458b-b2ad-4a3f808c36a0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.354222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.355111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.355393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.355673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] PATH_INFO: `/v3/projects/75286e1b5a6a4bff96baec252e9a00b0/users/9076f7899cb44bfa8d6484dd87238b35/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.371458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75286e1b5a6a4bff96baec252e9a00b0, user_id=9076f7899cb44bfa8d6484dd87238b35, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.374778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.387656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1013/2044] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:07 2022] PUT /identity/v3/projects/75286e1b5a6a4bff96baec252e9a00b0/users/9076f7899cb44bfa8d6484dd87238b35/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:07.399726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.400017 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.400345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.427993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] MFA Rules not processed for user `9076f7899cb44bfa8d6484dd87238b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:07.429269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:07.495471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1032/2045] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:07.508670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:07.510403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:07.542906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:07.588539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9275ed8-2b4f-4869-8304-7cf5d66b5e27 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:07.594018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8980985-4d4e-484e-a17d-facdcb9c3bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:07.594669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8980985-4d4e-484e-a17d-facdcb9c3bed service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:07.594958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8980985-4d4e-484e-a17d-facdcb9c3bed service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:07.595236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8980985-4d4e-484e-a17d-facdcb9c3bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:07.603574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8980985-4d4e-484e-a17d-facdcb9c3bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:07.605187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8980985-4d4e-484e-a17d-facdcb9c3bed service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:07.615091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1014/2046] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:17.492575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:17.610417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=22 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:18.033623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.093117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d0b6-9608-4b53-b82f-914c150c6adf None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:18.097774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:18.099388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.099667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.099940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] PATH_INFO: `/v3/users/9076f7899cb44bfa8d6484dd87238b35` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.101164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9076f7899cb44bfa8d6484dd87238b35)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:18.103117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:18.146940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9076f7899cb44bfa8d6484dd87238b35'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.153415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9076f7899cb44bfa8d6484dd87238b35'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.154114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9076f7899cb44bfa8d6484dd87238b35'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.154740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9076f7899cb44bfa8d6484dd87238b35'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.167207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9076f7899cb44bfa8d6484dd87238b35'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.179504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1033/2047] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:18 2022] DELETE /identity/v3/users/9076f7899cb44bfa8d6484dd87238b35 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:18.184183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.184478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.184761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.201564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:18.202761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.261959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1015/2048] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:18.271698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.273129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.329872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99963d1d-fba3-4723-b3d8-4939918fbe97 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:18.334249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99997687-840c-4e32-aa46-593c0d07f426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:18.334901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99997687-840c-4e32-aa46-593c0d07f426 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.335188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99997687-840c-4e32-aa46-593c0d07f426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.335470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99997687-840c-4e32-aa46-593c0d07f426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.344625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99997687-840c-4e32-aa46-593c0d07f426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:18.346071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99997687-840c-4e32-aa46-593c0d07f426 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:18.356618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1034/2049] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:18.627140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.629159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.641721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c84a985-b605-45e7-a08a-275f4bc390b0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:18.647578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:18.648476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.648759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.649035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] PATH_INFO: `/v3/projects/75286e1b5a6a4bff96baec252e9a00b0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.650227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=75286e1b5a6a4bff96baec252e9a00b0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:18.652212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:18.699598 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75286e1b5a6a4bff96baec252e9a00b0', 'request_id': 'req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.705728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75286e1b5a6a4bff96baec252e9a00b0', 'request_id': 'req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.711744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75286e1b5a6a4bff96baec252e9a00b0', 'request_id': 'req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.724118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] The token cache is being invalidate because project 75286e1b5a6a4bff96baec252e9a00b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:18.724462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.725096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a08c3b5-1a7b-417b-abc3-97569c8abe8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:18.726634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1016/2050] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:18 2022] DELETE /identity/v3/projects/75286e1b5a6a4bff96baec252e9a00b0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:18.731469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.731931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.732029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.749945 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:18.751500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.771401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:18.825441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1035/2051] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:18.834491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.834779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.835051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.853557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:18.854820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:18.862742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1017/2052] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:18.867484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.877726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d5a65-b19f-4ff8-bb36-6e06cfc2b825 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:18.882414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:18.883076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.883357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.883632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.884572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:18.884883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:18.886499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:18.898806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1036/2053] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:27:18.903116 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.912435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c5adc-bea4-4e29-8c85-d2cff3c7f875 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:18.917038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:18.917687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:18.917969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:18.918244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:18.918947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:18.921536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:18.947756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1018/2054] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:18 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:27:18.951816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:18.961862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f0db6-12b1-4230-a03b-cf6335a0cf8a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.017091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.017672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.017953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.018227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.018912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.020904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.040149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1037/2055] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:18 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:19.044268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.055093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e501144-ca10-4919-9b24-810241269477 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.060971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.061666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.061954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.062234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.062982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.064667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.075310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1019/2056] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:19.079324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.089300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2206a760-7a47-417d-82b9-4ed188221dc2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.093910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26836b59-4530-47b3-b22f-c3f56ebe65cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.094631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836b59-4530-47b3-b22f-c3f56ebe65cc admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.094911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836b59-4530-47b3-b22f-c3f56ebe65cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.095187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836b59-4530-47b3-b22f-c3f56ebe65cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.095910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26836b59-4530-47b3-b22f-c3f56ebe65cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.098532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26836b59-4530-47b3-b22f-c3f56ebe65cc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.108853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1038/2057] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:19.112951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.124317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd96d83-83d5-4f92-ad42-6d9ca002ab4e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.129058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.130128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.130412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.130690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] PATH_INFO: `/v3/projects/d80cb95df84c40cd813d9655ca8af1a5/users/d039c44cdbbf4d4eb4c10d36efcc096d/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.144837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d80cb95df84c40cd813d9655ca8af1a5, user_id=d039c44cdbbf4d4eb4c10d36efcc096d, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.147479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.160916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1020/2058] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:19 2022] PUT /identity/v3/projects/d80cb95df84c40cd813d9655ca8af1a5/users/d039c44cdbbf4d4eb4c10d36efcc096d/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:19.171606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.171890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.172195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.198503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] MFA Rules not processed for user `d039c44cdbbf4d4eb4c10d36efcc096d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:19.199726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:19.263252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1039/2059] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:19.268173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.290020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:19.333729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099cfa08-138a-469f-a932-fe5f562a59a8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.338348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.339004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.339286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.339559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.340288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.342195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.371917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1021/2060] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:19 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:19.376108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.387436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73ae97-e184-40aa-9d8e-7157569e50b1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.401187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:19.449274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.449942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.450223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.450498 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.451196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.453197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.474521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1040/2061] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:19 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:19.478686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.489270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172c2963-4c55-4717-8579-4403cd504125 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.494026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.494692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.495004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.495287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.496072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.497979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.508816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1022/2062] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:19.513173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.524199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a135725-ec4d-4e61-90e5-70e8c8921340 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.528842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.529730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.530067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.530349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] PATH_INFO: `/v3/projects/e936cb21806f41b38fd3fee2a6da21de/users/83205368863344398b992fc82928eb89/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.546065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e936cb21806f41b38fd3fee2a6da21de, user_id=83205368863344398b992fc82928eb89, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.548914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.563911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1041/2063] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:19 2022] PUT /identity/v3/projects/e936cb21806f41b38fd3fee2a6da21de/users/83205368863344398b992fc82928eb89/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:19.568298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.578355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e588a73b-ee99-47a6-acc5-8bb78e3d57a6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.636701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17487966-0bbf-42b9-b03f-97172e9d94b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.637366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17487966-0bbf-42b9-b03f-97172e9d94b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.637635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17487966-0bbf-42b9-b03f-97172e9d94b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.637916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17487966-0bbf-42b9-b03f-97172e9d94b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.638635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17487966-0bbf-42b9-b03f-97172e9d94b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.640266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17487966-0bbf-42b9-b03f-97172e9d94b5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.649922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1023/2064] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:27:19.654110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.664398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc007281-38be-4452-b813-c64daef63a7a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.668939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.669818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.670100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.670373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] PATH_INFO: `/v3/projects/e936cb21806f41b38fd3fee2a6da21de/users/83205368863344398b992fc82928eb89/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.672398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e936cb21806f41b38fd3fee2a6da21de, user_id=83205368863344398b992fc82928eb89, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.675050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.685587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1042/2065] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:19 2022] PUT /identity/v3/projects/e936cb21806f41b38fd3fee2a6da21de/users/83205368863344398b992fc82928eb89/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:19.696199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.696496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.696779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.722572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] MFA Rules not processed for user `83205368863344398b992fc82928eb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:19.724628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:19.792091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1024/2066] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:19.803919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.805657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:19.862319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221367c1-1f9b-4c0b-874a-8de5982cb83a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:19.866550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e79927-8434-4ae2-b23e-87e4f840fac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:19.867196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79927-8434-4ae2-b23e-87e4f840fac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:19.867473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79927-8434-4ae2-b23e-87e4f840fac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:19.867749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e79927-8434-4ae2-b23e-87e4f840fac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:19.877014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e79927-8434-4ae2-b23e-87e4f840fac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:19.878439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e79927-8434-4ae2-b23e-87e4f840fac1 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:19.887712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1043/2067] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:19.952740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:19.970348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:20.017047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2822f2cd-458d-47da-abee-dd90ee1ba854 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:20.022040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:20.023082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:20.023192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:20.023441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] PATH_INFO: `/v3/users/d039c44cdbbf4d4eb4c10d36efcc096d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:20.024777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d039c44cdbbf4d4eb4c10d36efcc096d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:20.026778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:20.075467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd039c44cdbbf4d4eb4c10d36efcc096d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.084220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd039c44cdbbf4d4eb4c10d36efcc096d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.084939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd039c44cdbbf4d4eb4c10d36efcc096d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.085762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd039c44cdbbf4d4eb4c10d36efcc096d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.094754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd039c44cdbbf4d4eb4c10d36efcc096d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.111209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1025/2068] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:19 2022] DELETE /identity/v3/users/d039c44cdbbf4d4eb4c10d36efcc096d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:20.116582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:20.116872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:20.117149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:20.134682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:20.135871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:20.193487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1044/2069] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:20.203099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:20.204766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:20.264875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979799c-94ce-472b-a448-26b9d66b8cff admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:20.268517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:20.269185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:20.269474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:20.269775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:20.278932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:20.280340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:20.289817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1026/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:20.548372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:20.549845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:20.563207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d4274-08a2-4a83-9dca-44dbfd8a5563 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:20.568003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:20.568932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:20.569219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:20.569501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] PATH_INFO: `/v3/projects/d80cb95df84c40cd813d9655ca8af1a5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:20.570709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d80cb95df84c40cd813d9655ca8af1a5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:20.572785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:20.581219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:20.626974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd80cb95df84c40cd813d9655ca8af1a5', 'request_id': 'req-8e667143-9f35-4bb4-9956-3f47da163b50'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.637091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd80cb95df84c40cd813d9655ca8af1a5', 'request_id': 'req-8e667143-9f35-4bb4-9956-3f47da163b50'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.643245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd80cb95df84c40cd813d9655ca8af1a5', 'request_id': 'req-8e667143-9f35-4bb4-9956-3f47da163b50'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.653431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] The token cache is being invalidate because project d80cb95df84c40cd813d9655ca8af1a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:20.653714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.654322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.655846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1045/2071] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:20 2022] DELETE /identity/v3/projects/d80cb95df84c40cd813d9655ca8af1a5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:20.659913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:20.670856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437968e6-b474-4658-97ad-6eb794f5607b service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:20.704930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:20.730582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:20.731459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:20.731745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:20.732025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] PATH_INFO: `/v3/users/83205368863344398b992fc82928eb89` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:20.733248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83205368863344398b992fc82928eb89)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:20.735176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:20.783816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83205368863344398b992fc82928eb89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.792058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83205368863344398b992fc82928eb89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.793074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83205368863344398b992fc82928eb89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.794083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83205368863344398b992fc82928eb89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.802627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f1922821-4a93-49ee-a436-0432dcf9f522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83205368863344398b992fc82928eb89'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:20.819651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1027/2072] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:20 2022] DELETE /identity/v3/users/83205368863344398b992fc82928eb89 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:21.091464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.093053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.120819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e667143-9f35-4bb4-9956-3f47da163b50 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.180969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.181835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.182115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.182388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] PATH_INFO: `/v3/projects/e936cb21806f41b38fd3fee2a6da21de` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.183559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e936cb21806f41b38fd3fee2a6da21de)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.185513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.233304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e936cb21806f41b38fd3fee2a6da21de', 'request_id': 'req-cc59f53b-8135-4c93-9b49-8819eec9e0ec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:21.241872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e936cb21806f41b38fd3fee2a6da21de', 'request_id': 'req-cc59f53b-8135-4c93-9b49-8819eec9e0ec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:21.248565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e936cb21806f41b38fd3fee2a6da21de', 'request_id': 'req-cc59f53b-8135-4c93-9b49-8819eec9e0ec'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:21.259787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] The token cache is being invalidate because project e936cb21806f41b38fd3fee2a6da21de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:21.260095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:21.260754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cc59f53b-8135-4c93-9b49-8819eec9e0ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:21.262287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1046/2073] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:21 2022] DELETE /identity/v3/projects/e936cb21806f41b38fd3fee2a6da21de => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:21.267076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.267390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.267665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.285602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:21.286795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.345437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1028/2074] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:21.355741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.356036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.356373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.374926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:21.376159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.384982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1047/2075] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:21.389105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.399333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225b42-8886-44dd-8de9-f32119d00799 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.403793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.404474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.404755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.405026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.405754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:21.406056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.407645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.419307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1029/2076] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:27:21.423321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.432399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d5d38-2ed2-47c0-8a9a-ae61b0206eb3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.436933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.437570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.437852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.438129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.438805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.440743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.460266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:21.469061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1048/2077] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:21.473021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.483093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599cb09-3c67-4e7a-a043-eaa3aef5c37a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.525000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:21.541413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.542080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.542365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.542644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.543344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.545310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.565150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1030/2078] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:21.569816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.580210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e6cfbe-9f57-4643-b40c-cc2b89e4d1a0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.585111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05395558-3d80-45df-8658-13a564067003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.585765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05395558-3d80-45df-8658-13a564067003 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.586048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05395558-3d80-45df-8658-13a564067003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.586327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05395558-3d80-45df-8658-13a564067003 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.587049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05395558-3d80-45df-8658-13a564067003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.588789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05395558-3d80-45df-8658-13a564067003 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.600695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1049/2079] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:21.604578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.615070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284f494-66b1-4532-841a-bf03028f883d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.619673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1c19-a404-4dfc-83e4-6d3462c5f644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.620363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1c19-a404-4dfc-83e4-6d3462c5f644 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.620645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1c19-a404-4dfc-83e4-6d3462c5f644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.620921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1c19-a404-4dfc-83e4-6d3462c5f644 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.621915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb1c19-a404-4dfc-83e4-6d3462c5f644 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.623499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb1c19-a404-4dfc-83e4-6d3462c5f644 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.634133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1031/2080] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:21.638152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05395558-3d80-45df-8658-13a564067003 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.648539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05395558-3d80-45df-8658-13a564067003 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.653091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.653965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.654243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.654515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] PATH_INFO: `/v3/projects/542853068a3e4eedb2acb4acd6b9c26f/users/7182478863d04c0b9d716686a8f08028/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.669529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542853068a3e4eedb2acb4acd6b9c26f, user_id=7182478863d04c0b9d716686a8f08028, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.672435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.685279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1050/2081] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:21 2022] PUT /identity/v3/projects/542853068a3e4eedb2acb4acd6b9c26f/users/7182478863d04c0b9d716686a8f08028/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:21.695391 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.695893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.696003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.722467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] MFA Rules not processed for user `7182478863d04c0b9d716686a8f08028`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:21.724628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:21.789069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1032/2082] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:21.793515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.854080 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff4775-74b6-4f28-b156-8026dfb35f27 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.858695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.859346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.859632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.859911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.860626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.862531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.887134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1051/2083] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/projects => generated 391 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:21.891378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.902868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa441d-78f2-4476-a4ea-dd2d9353f5b1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:21.960360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:21.961052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:21.961326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:21.961603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:21.962329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:21.964292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:21.983878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1033/2084] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:21 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:21.987785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:21.998440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10832a48-b784-498b-b4d2-04a92bb81454 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:22.003107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:22.003758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.004610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.004895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.005618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:22.007224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:22.018511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1052/2085] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:22.023397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:22.033167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3bf49-9985-4f85-8093-669153c53a78 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:22.037934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:22.038802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.039082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.039353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] PATH_INFO: `/v3/projects/c1a03c5b886b4bfa97051625081f2d48/users/88050cb62d3449639b76730c3e11b921/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.053638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a03c5b886b4bfa97051625081f2d48, user_id=88050cb62d3449639b76730c3e11b921, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:22.056315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:22.069588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1034/2086] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:22 2022] PUT /identity/v3/projects/c1a03c5b886b4bfa97051625081f2d48/users/88050cb62d3449639b76730c3e11b921/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:22.073543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:22.084744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa90640-1676-4f98-8b99-25f501eee2f2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:22.139819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a56162-6d52-40aa-9858-a68c50eb3fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:22.140513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a56162-6d52-40aa-9858-a68c50eb3fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.140795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a56162-6d52-40aa-9858-a68c50eb3fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.141070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a56162-6d52-40aa-9858-a68c50eb3fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.141787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a56162-6d52-40aa-9858-a68c50eb3fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:22.143386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a56162-6d52-40aa-9858-a68c50eb3fa6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:22.153566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1053/2087] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:22.157805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:22.168228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e03561-985c-42c8-b5de-aea410ba7107 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:22.172820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:22.173695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.173975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.174248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] PATH_INFO: `/v3/projects/c1a03c5b886b4bfa97051625081f2d48/users/88050cb62d3449639b76730c3e11b921/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.176331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1a03c5b886b4bfa97051625081f2d48, user_id=88050cb62d3449639b76730c3e11b921, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:22.178964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:22.189824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1035/2088] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:22 2022] PUT /identity/v3/projects/c1a03c5b886b4bfa97051625081f2d48/users/88050cb62d3449639b76730c3e11b921/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:22.201151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.201564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.201934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.229126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] MFA Rules not processed for user `88050cb62d3449639b76730c3e11b921`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:22.230317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:22.250174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:22.306408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1054/2089] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:22.316956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:22.318656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:22.325218 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:22.389444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0891ae-52a6-479f-bf67-4e12536f44a4 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:22.393542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:22.394202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.394552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.395581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.407774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:22.409484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:22.420884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1036/2090] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:22.468167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:22.469646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:22.482400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf5311-4f04-4448-bdca-7c451308dc16 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:22.486607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe13d51-86f3-49cb-8868-09f10fa16168 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:22.486805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13d51-86f3-49cb-8868-09f10fa16168 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:22.487087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13d51-86f3-49cb-8868-09f10fa16168 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:22.487361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe13d51-86f3-49cb-8868-09f10fa16168 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:22.546196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13d51-86f3-49cb-8868-09f10fa16168 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:22.547648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe13d51-86f3-49cb-8868-09f10fa16168 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:22.557197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1055/2091] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:26.849864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:26.911565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21864715-1c2e-42ac-a143-31c98fae0dab service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:26.916205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:26.917106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:26.917393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:26.917671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] PATH_INFO: `/v3/users/7182478863d04c0b9d716686a8f08028` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:26.918877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7182478863d04c0b9d716686a8f08028)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:26.920853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:26.967080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7182478863d04c0b9d716686a8f08028'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:26.974649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7182478863d04c0b9d716686a8f08028'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:26.975351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7182478863d04c0b9d716686a8f08028'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:26.975966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7182478863d04c0b9d716686a8f08028'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:26.982866 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7182478863d04c0b9d716686a8f08028'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.000261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1037/2092] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:26 2022] DELETE /identity/v3/users/7182478863d04c0b9d716686a8f08028 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:27.004816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.005270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.005375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.022592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:27.023777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:27.079906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1056/2093] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:27.090245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:27.091756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:27.124045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:27.154519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4994015-ec8e-4d17-8481-a06c9a00e17c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:27.158209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:27.158863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.159154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.159435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.169206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:27.170654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:27.180416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1038/2094] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:27.465299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:27.466771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:27.487453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512fcaa-54eb-48d3-8650-5df9505cbcfc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:27.504937 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:27.506113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.506379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.506672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] PATH_INFO: `/v3/projects/542853068a3e4eedb2acb4acd6b9c26f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.507873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=542853068a3e4eedb2acb4acd6b9c26f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:27.509851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:27.570820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542853068a3e4eedb2acb4acd6b9c26f', 'request_id': 'req-ff4a1ae9-33f3-4652-89c0-e1b385066439'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.581535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:27.585580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542853068a3e4eedb2acb4acd6b9c26f', 'request_id': 'req-ff4a1ae9-33f3-4652-89c0-e1b385066439'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.595607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542853068a3e4eedb2acb4acd6b9c26f', 'request_id': 'req-ff4a1ae9-33f3-4652-89c0-e1b385066439'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.608772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] The token cache is being invalidate because project 542853068a3e4eedb2acb4acd6b9c26f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:27.609065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.609740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.611295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1057/2095] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:27 2022] DELETE /identity/v3/projects/542853068a3e4eedb2acb4acd6b9c26f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:27.615435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:27.625773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696fa5-f363-4a9a-8fd1-3c16828b6c87 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:27.688347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:27.689148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:27.689434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:27.689713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] PATH_INFO: `/v3/users/88050cb62d3449639b76730c3e11b921` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:27.690916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] RBAC: Authorizing `identity:delete_user(user_id=88050cb62d3449639b76730c3e11b921)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:27.692942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:27.743925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88050cb62d3449639b76730c3e11b921'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.753493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88050cb62d3449639b76730c3e11b921'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.754841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88050cb62d3449639b76730c3e11b921'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.755474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88050cb62d3449639b76730c3e11b921'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.768282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3493b27b-e990-4b7f-98df-3a22cd14713e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88050cb62d3449639b76730c3e11b921'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:27.789533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1039/2096] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:27 2022] DELETE /identity/v3/users/88050cb62d3449639b76730c3e11b921 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:28.063743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.065555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:28.082007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1ae9-33f3-4652-89c0-e1b385066439 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.143509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.145926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.146214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.146501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] PATH_INFO: `/v3/projects/c1a03c5b886b4bfa97051625081f2d48` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.147820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1a03c5b886b4bfa97051625081f2d48)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.149873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.198180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1a03c5b886b4bfa97051625081f2d48', 'request_id': 'req-d21ccc0d-d787-4363-98ed-0890a46f4413'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:28.205346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1a03c5b886b4bfa97051625081f2d48', 'request_id': 'req-d21ccc0d-d787-4363-98ed-0890a46f4413'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:28.212074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1a03c5b886b4bfa97051625081f2d48', 'request_id': 'req-d21ccc0d-d787-4363-98ed-0890a46f4413'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:28.222381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] The token cache is being invalidate because project c1a03c5b886b4bfa97051625081f2d48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:28.222701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:28.223366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d21ccc0d-d787-4363-98ed-0890a46f4413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:28.224953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1058/2097] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:28 2022] DELETE /identity/v3/projects/c1a03c5b886b4bfa97051625081f2d48 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:28.230174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.230629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.230744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.249888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:28.251068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:28.304117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:28.318943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1040/2098] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:28.326708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.327023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.327304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.345922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:28.347119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:28.355469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1059/2099] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:28.359316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.370543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67e0c69-cf0f-4deb-9347-45ad4e335a60 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.375338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.375987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.376308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.376593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.377326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:28.377636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.379973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.391483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1041/2100] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:27:28.395832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.405122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e92f-7cca-4fa0-af79-b86ad4d88f6c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.409795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.410570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.410858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.411134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.411830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.413780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.439035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1060/2101] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:28.443684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.453502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de51c286-2c5e-4dfc-8ce5-ea58e26bf6a3 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.509380 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.510044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.510325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.510601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.511296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.513257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.532702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1042/2102] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:28.536735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.546812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b27c8-1756-4400-a2c0-da2f77778f2c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.551996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.552677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.552959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.553235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.553956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.555559 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.565788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1061/2103] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:28.570308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.580222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960367fc-d901-4580-bf33-03884f5c4ff1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.585356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f07be4-a9c9-4c0c-9e16-88ca45a25dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.586342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f07be4-a9c9-4c0c-9e16-88ca45a25dfc admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.586342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f07be4-a9c9-4c0c-9e16-88ca45a25dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.586554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f07be4-a9c9-4c0c-9e16-88ca45a25dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.587283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f07be4-a9c9-4c0c-9e16-88ca45a25dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.589010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f07be4-a9c9-4c0c-9e16-88ca45a25dfc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.600543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1043/2104] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:28.604596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.615477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748edde2-63df-4f23-a856-030b4a60d5af admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.621144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.622294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.622576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.622849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] PATH_INFO: `/v3/projects/1b2843be72e9404894459c420a72c3aa/users/2cd7a914dc544a2f8bcb718e5cbb66da/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.644050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b2843be72e9404894459c420a72c3aa, user_id=2cd7a914dc544a2f8bcb718e5cbb66da, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.647195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.660880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1062/2105] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:28 2022] PUT /identity/v3/projects/1b2843be72e9404894459c420a72c3aa/users/2cd7a914dc544a2f8bcb718e5cbb66da/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:28.672911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.673206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.673491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.700836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] MFA Rules not processed for user `2cd7a914dc544a2f8bcb718e5cbb66da`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:28.702035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:28.765821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1044/2106] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:28.770349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.779344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:28.834339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1019e25-f23b-4e4c-8eed-958263de5b8a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.839246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.839905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.840220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.840501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.841197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.843141 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.868401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1063/2107] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:28.872619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.883922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfd563-af2a-406c-8b0f-c97a72833e0a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.926660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:28.943798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.944573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.944858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.945268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.945983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.947982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:28.967739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1045/2108] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:28 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:28.971754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:28.982922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5522466d-980f-41f5-b5f4-b747674ad4a6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:28.987507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:28.988830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:28.989288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:28.989565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:28.990290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:28.991924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.002955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1064/2109] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:29.007049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.017355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937e3a8-e305-49b7-b41c-1d8100f75e08 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.022280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a8506-df39-4032-9b14-d94b3e6640d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.022958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8506-df39-4032-9b14-d94b3e6640d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.023234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8506-df39-4032-9b14-d94b3e6640d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.023575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a8506-df39-4032-9b14-d94b3e6640d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.024348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a8506-df39-4032-9b14-d94b3e6640d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.025943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a8506-df39-4032-9b14-d94b3e6640d2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.036765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1046/2110] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:29.040868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.051405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e8f26-174a-45b0-bd0c-c1aefbb9585b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.056147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.057030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.057314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.057597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] PATH_INFO: `/v3/projects/fa6c0f4bcb7f46d4a14a6deaeb237ab4/users/0359ae9682494d8697df78d2e39ce725/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.071916 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa6c0f4bcb7f46d4a14a6deaeb237ab4, user_id=0359ae9682494d8697df78d2e39ce725, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.074624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.087477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1065/2111] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:29 2022] PUT /identity/v3/projects/fa6c0f4bcb7f46d4a14a6deaeb237ab4/users/0359ae9682494d8697df78d2e39ce725/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:29.099071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.099360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.099639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.126603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] MFA Rules not processed for user `0359ae9682494d8697df78d2e39ce725`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:29.127807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:29.197698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1047/2112] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:29.202268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.263097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5907062b-c4e1-4ef6-9002-e4c60b7a3aa3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.267791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.268632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.268917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.269194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.270394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.272331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.298432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1066/2113] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:29 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:29.302540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.313302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18766a-8dee-4487-bdbb-31479b27bbad None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.370044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.370715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.371003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.371282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.372280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.374224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.393124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1048/2114] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:29 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:27:29.397778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.407913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8408f1-30b2-4416-8ca3-d41080e0e316 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.413272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.413925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.414207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.414484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.415213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.416861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.427556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1067/2115] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:29.431542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.441402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64dadb8-873a-4eba-828c-10e65c82a2e0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.446233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.447102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.447386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.447661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] PATH_INFO: `/v3/projects/c472aa4c95db427db80e48819f14b41d/users/72d02a52146a4de09d14d0cc6032412b/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.463938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] RBAC: Authorizing `identity:create_grant(project_id=c472aa4c95db427db80e48819f14b41d, user_id=72d02a52146a4de09d14d0cc6032412b, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.466629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.479467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1049/2116] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:29 2022] PUT /identity/v3/projects/c472aa4c95db427db80e48819f14b41d/users/72d02a52146a4de09d14d0cc6032412b/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:29.483267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.493207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4272305-3e7a-45d4-b0fc-6ffa8f914509 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.523189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:29.551327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d7767-9bfb-4258-828a-a1c56dc9c3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.551986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7767-9bfb-4258-828a-a1c56dc9c3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.552295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7767-9bfb-4258-828a-a1c56dc9c3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.552574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d7767-9bfb-4258-828a-a1c56dc9c3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.553289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d7767-9bfb-4258-828a-a1c56dc9c3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.554919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d7767-9bfb-4258-828a-a1c56dc9c3b9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.564355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1068/2117] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:29.569556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.579421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d0ba1-3e40-4ef5-8fb3-2747c9f60ced admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.584667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.585534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.585813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.586087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] PATH_INFO: `/v3/projects/c472aa4c95db427db80e48819f14b41d/users/72d02a52146a4de09d14d0cc6032412b/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.588019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c472aa4c95db427db80e48819f14b41d, user_id=72d02a52146a4de09d14d0cc6032412b, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.590684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.600872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1050/2118] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:29 2022] PUT /identity/v3/projects/c472aa4c95db427db80e48819f14b41d/users/72d02a52146a4de09d14d0cc6032412b/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:29.610889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.611182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.611462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.637969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] MFA Rules not processed for user `72d02a52146a4de09d14d0cc6032412b`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:29.639164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:29.708764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1069/2119] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:29.719558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.721284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:29.727902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:29.781662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760aecad-7f1d-48c8-a938-41ef966e11b4 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.785672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.786318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.786623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.786902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.795892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.797356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.806823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1051/2120] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:29.845812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:29.847183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:29.858967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f75a2dc-1933-45a6-9b58-91262f0646c5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:29.862576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdd6a4-ac36-4b20-bf98-ef1762b0382f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:29.863829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdd6a4-ac36-4b20-bf98-ef1762b0382f service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:29.864143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdd6a4-ac36-4b20-bf98-ef1762b0382f service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:29.864422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdd6a4-ac36-4b20-bf98-ef1762b0382f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:29.920071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdd6a4-ac36-4b20-bf98-ef1762b0382f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:29.921539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdd6a4-ac36-4b20-bf98-ef1762b0382f service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:29.930546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1070/2121] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:32.780922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:32.844336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b3f74-6282-4f3e-87d0-8fa88ee4df6d service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:32.849572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:32.850465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:32.850747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:32.851020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] PATH_INFO: `/v3/users/2cd7a914dc544a2f8bcb718e5cbb66da` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:32.852235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd7a914dc544a2f8bcb718e5cbb66da)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:32.854147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:32.899429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd7a914dc544a2f8bcb718e5cbb66da'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:32.907194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd7a914dc544a2f8bcb718e5cbb66da'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:32.907909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd7a914dc544a2f8bcb718e5cbb66da'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:32.908638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd7a914dc544a2f8bcb718e5cbb66da'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:32.916374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd7a914dc544a2f8bcb718e5cbb66da'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:32.933259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1052/2122] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:32 2022] DELETE /identity/v3/users/2cd7a914dc544a2f8bcb718e5cbb66da => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:32.937746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:32.938174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:32.938333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:32.955360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:32.956539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:33.002065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:33.017343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1071/2123] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:33.027107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:33.028609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:33.059459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:33.087466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608502-773c-4070-83d6-1ca7946cf3cb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:33.091060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:33.091706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:33.092002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:33.092301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:33.101532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:33.103049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:33.112595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1053/2124] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:33.383879 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:33.385407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:33.397873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0d158-d5f7-4738-90d3-54fe72326989 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:33.402460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:33.403330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:33.403616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:33.403922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] PATH_INFO: `/v3/projects/1b2843be72e9404894459c420a72c3aa` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:33.405585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b2843be72e9404894459c420a72c3aa)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:33.407489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:33.456380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b2843be72e9404894459c420a72c3aa', 'request_id': 'req-53f46687-5709-49d5-9df2-f9352cf3cda9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.464121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b2843be72e9404894459c420a72c3aa', 'request_id': 'req-53f46687-5709-49d5-9df2-f9352cf3cda9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.470681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b2843be72e9404894459c420a72c3aa', 'request_id': 'req-53f46687-5709-49d5-9df2-f9352cf3cda9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.482955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] The token cache is being invalidate because project 1b2843be72e9404894459c420a72c3aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:33.483244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.483886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.485486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1072/2125] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:33 2022] DELETE /identity/v3/projects/1b2843be72e9404894459c420a72c3aa => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:33.490139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:33.501432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d562bb2e-bffb-4775-b8da-9246b235b81e service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:33.559035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:33.560047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:33.560485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:33.560778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] PATH_INFO: `/v3/users/0359ae9682494d8697df78d2e39ce725` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:33.561999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0359ae9682494d8697df78d2e39ce725)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:33.564206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:33.614482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0359ae9682494d8697df78d2e39ce725'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.622292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0359ae9682494d8697df78d2e39ce725'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.622905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0359ae9682494d8697df78d2e39ce725'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.623547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0359ae9682494d8697df78d2e39ce725'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.631399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0359ae9682494d8697df78d2e39ce725'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:33.649669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1054/2126] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:33 2022] DELETE /identity/v3/users/0359ae9682494d8697df78d2e39ce725 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:33.938303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:33.939795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:33.953788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f46687-5709-49d5-9df2-f9352cf3cda9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:34.018813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:34.020608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.020890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.021164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] PATH_INFO: `/v3/projects/fa6c0f4bcb7f46d4a14a6deaeb237ab4` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.022403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa6c0f4bcb7f46d4a14a6deaeb237ab4)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:34.024374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:34.074779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa6c0f4bcb7f46d4a14a6deaeb237ab4', 'request_id': 'req-9190cef0-9f22-43ad-8395-0ad9e08f4da9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.081476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa6c0f4bcb7f46d4a14a6deaeb237ab4', 'request_id': 'req-9190cef0-9f22-43ad-8395-0ad9e08f4da9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.088911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa6c0f4bcb7f46d4a14a6deaeb237ab4', 'request_id': 'req-9190cef0-9f22-43ad-8395-0ad9e08f4da9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.099622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] The token cache is being invalidate because project fa6c0f4bcb7f46d4a14a6deaeb237ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:34.099911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.100594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.102233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1073/2127] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:33 2022] DELETE /identity/v3/projects/fa6c0f4bcb7f46d4a14a6deaeb237ab4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:34.106265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:34.107680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:34.120540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd041872-5321-41a7-a2ae-42e9f5df6ee9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:34.164512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:34.177930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:34.178791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.179071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.179346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] PATH_INFO: `/v3/users/72d02a52146a4de09d14d0cc6032412b` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.180553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d02a52146a4de09d14d0cc6032412b)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:34.182507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:34.227387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d02a52146a4de09d14d0cc6032412b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.234638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d02a52146a4de09d14d0cc6032412b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.235277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d02a52146a4de09d14d0cc6032412b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.235887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d02a52146a4de09d14d0cc6032412b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.242863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3c0956e5-1022-466b-879d-9295a472b604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d02a52146a4de09d14d0cc6032412b'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.260936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1055/2128] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:34 2022] DELETE /identity/v3/users/72d02a52146a4de09d14d0cc6032412b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:34.539998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:34.541522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:34.551208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:34.555916 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190cef0-9f22-43ad-8395-0ad9e08f4da9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:34.618634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:34.619499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.619775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.620050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] PATH_INFO: `/v3/projects/c472aa4c95db427db80e48819f14b41d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.621261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c472aa4c95db427db80e48819f14b41d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:34.623433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:34.671285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c472aa4c95db427db80e48819f14b41d', 'request_id': 'req-4c385e21-dbcc-4eed-b64e-c96fd05b4728'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.678860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c472aa4c95db427db80e48819f14b41d', 'request_id': 'req-4c385e21-dbcc-4eed-b64e-c96fd05b4728'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.685105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c472aa4c95db427db80e48819f14b41d', 'request_id': 'req-4c385e21-dbcc-4eed-b64e-c96fd05b4728'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.695373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] The token cache is being invalidate because project c472aa4c95db427db80e48819f14b41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:34.695658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.696300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4c385e21-dbcc-4eed-b64e-c96fd05b4728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:34.697834 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1074/2129] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:34 2022] DELETE /identity/v3/projects/c472aa4c95db427db80e48819f14b41d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:34.704190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.704190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.704389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.723567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:34.724717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:34.786465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1056/2130] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:34.794855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.795144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.795421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.814688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:34.815532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:34.825328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1075/2131] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:34.829843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:34.840017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600d0a0-0b01-415f-80bb-505ca39feb87 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:34.844657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:34.845325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.845628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.845885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.846622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:34.846931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:34.848963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:34.864873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1057/2132] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:27:34.869312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:34.880014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd7b9a-3b0d-40e7-b3a2-9903de7c7a0b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:34.884715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:34.885368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.885653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.885933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.886629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:34.888600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:34.915731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1076/2133] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:34 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:34.919930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:34.930347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fd8ca-adfc-4aaf-87e7-f608601cee8b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:34.982279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:34.991266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:34.992119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:34.992430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:34.993166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:34.993832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:34.995795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.020138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1058/2134] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:34 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:35.022245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.032894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed62083-f4a5-4d01-917c-b10eb4bf125f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.038632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.039233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.039516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.039795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.040549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.042160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.052957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1077/2135] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:27:35.056963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.069499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520948ec-8e1b-4862-afcb-ff75742303df admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.074813 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897c615-e759-4b90-8e4a-36bb08d047c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.075477 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c615-e759-4b90-8e4a-36bb08d047c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.075789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c615-e759-4b90-8e4a-36bb08d047c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.076063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897c615-e759-4b90-8e4a-36bb08d047c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.076817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c615-e759-4b90-8e4a-36bb08d047c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.078421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897c615-e759-4b90-8e4a-36bb08d047c7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.088738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1059/2136] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:27:35.093459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.105798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0aac86-21bf-47a7-ae17-68ad5a12e819 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.110807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.111862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.112248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.112545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] PATH_INFO: `/v3/projects/90bac1a1ac8f45a796e713e67d269cbb/users/2eb0ebfda1b447c9a96c9ed52742fcb7/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.130496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=90bac1a1ac8f45a796e713e67d269cbb, user_id=2eb0ebfda1b447c9a96c9ed52742fcb7, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.133239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.147117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1078/2137] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:35 2022] PUT /identity/v3/projects/90bac1a1ac8f45a796e713e67d269cbb/users/2eb0ebfda1b447c9a96c9ed52742fcb7/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:35.159166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.159633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.159837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.190220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] MFA Rules not processed for user `2eb0ebfda1b447c9a96c9ed52742fcb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:35.191504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:35.263363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1060/2138] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:35.267794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.301459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:35.330688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238721-b81c-4483-8141-28a36f053d8a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.336190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.336837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.337124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.337403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.338104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.340071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.365819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1079/2139] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:35 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:35.369930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.381317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbf0fe-ef3a-43ad-bfd8-33824db11030 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.443990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.444675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.444956 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.445230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.445913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.447856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.466893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1061/2140] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:35 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:27:35.470918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.482023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738b440-ab55-4020-8b79-0b0d4f5033ff admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.486568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.487215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.487496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.487780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.488556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.490148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.500728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1080/2141] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:35.505665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.515491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cf7d9-367b-48d9-b21d-cf91ca70c360 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.520717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.521585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.521860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.522141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] PATH_INFO: `/v3/projects/41ed2da01e134de38060ecf348bc159c/users/61aee57a79b44f328e876d1b04268e6c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.536541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ed2da01e134de38060ecf348bc159c, user_id=61aee57a79b44f328e876d1b04268e6c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.539195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.551384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1062/2142] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:35 2022] PUT /identity/v3/projects/41ed2da01e134de38060ecf348bc159c/users/61aee57a79b44f328e876d1b04268e6c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:35.555462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.565327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07121d48-e74d-4859-a0b4-13206c625793 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.621132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e1b75-8f6b-4551-b1ea-d315db309325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.622061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e1b75-8f6b-4551-b1ea-d315db309325 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.622347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e1b75-8f6b-4551-b1ea-d315db309325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.622654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e1b75-8f6b-4551-b1ea-d315db309325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.623339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e1b75-8f6b-4551-b1ea-d315db309325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.624974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e1b75-8f6b-4551-b1ea-d315db309325 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.637390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1081/2143] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:35.641686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.652833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46a77c-2847-409a-8006-3aa55be8f3ea admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.657456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.658358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.658642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.658919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] PATH_INFO: `/v3/projects/41ed2da01e134de38060ecf348bc159c/users/61aee57a79b44f328e876d1b04268e6c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.661021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ed2da01e134de38060ecf348bc159c, user_id=61aee57a79b44f328e876d1b04268e6c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.663672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.674183 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1063/2144] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:35 2022] PUT /identity/v3/projects/41ed2da01e134de38060ecf348bc159c/users/61aee57a79b44f328e876d1b04268e6c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:35.684956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.685248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.685527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.713418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] MFA Rules not processed for user `61aee57a79b44f328e876d1b04268e6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:35.714644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:35.782833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1082/2145] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:35 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:35.794452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:35.796222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:35.815033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:35.855638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee56164-99e8-4363-806a-b0ebd6eb2657 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:35.859276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622b9cf-d8a9-40a2-bb7e-2c29e9741c01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:35.859919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622b9cf-d8a9-40a2-bb7e-2c29e9741c01 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:35.860248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622b9cf-d8a9-40a2-bb7e-2c29e9741c01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:35.860528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622b9cf-d8a9-40a2-bb7e-2c29e9741c01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:35.870635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622b9cf-d8a9-40a2-bb7e-2c29e9741c01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:35.872375 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d622b9cf-d8a9-40a2-bb7e-2c29e9741c01 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:35.882759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1064/2146] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:35 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:36.549138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:36.580259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:36.614316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db194c-146f-412e-a819-f9bd233a9c57 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:36.618747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:36.619616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:36.619902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:36.620210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] PATH_INFO: `/v3/users/2eb0ebfda1b447c9a96c9ed52742fcb7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:36.621644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb0ebfda1b447c9a96c9ed52742fcb7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:36.623584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:36.667271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb0ebfda1b447c9a96c9ed52742fcb7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:36.673708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb0ebfda1b447c9a96c9ed52742fcb7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:36.674260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb0ebfda1b447c9a96c9ed52742fcb7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:36.674863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb0ebfda1b447c9a96c9ed52742fcb7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:36.687070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb0ebfda1b447c9a96c9ed52742fcb7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:36.699110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1083/2147] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:36 2022] DELETE /identity/v3/users/2eb0ebfda1b447c9a96c9ed52742fcb7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:36.703616 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:36.704102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:36.704194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:36.722921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:36.724133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:36.781739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1065/2148] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:27:36.791616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:36.793048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:36.849944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c55c27-ddec-4f61-a761-4304c3a003ed admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:36.853596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:36.854249 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:36.854531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:36.854821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:36.864557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:36.865996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:36.875655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1084/2149] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:37.143741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:37.145258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:37.162649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da1fa-2f7d-4394-a6ff-af642e3a5839 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:37.176739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:37.177659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:37.177942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:37.178217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] PATH_INFO: `/v3/projects/90bac1a1ac8f45a796e713e67d269cbb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:37.179474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90bac1a1ac8f45a796e713e67d269cbb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:37.181443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:37.229235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90bac1a1ac8f45a796e713e67d269cbb', 'request_id': 'req-27e09754-48ab-4c81-83c4-632b7981d4e0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.235255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90bac1a1ac8f45a796e713e67d269cbb', 'request_id': 'req-27e09754-48ab-4c81-83c4-632b7981d4e0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.241481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90bac1a1ac8f45a796e713e67d269cbb', 'request_id': 'req-27e09754-48ab-4c81-83c4-632b7981d4e0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.253987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] The token cache is being invalidate because project 90bac1a1ac8f45a796e713e67d269cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:37.254327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.254959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.256655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1066/2150] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:37 2022] DELETE /identity/v3/projects/90bac1a1ac8f45a796e713e67d269cbb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:37.260668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:37.270089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26497881-098b-4812-b381-e42e9f932a5b service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:37.314941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:37.328175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:37.329050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:37.329332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:37.329606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] PATH_INFO: `/v3/users/61aee57a79b44f328e876d1b04268e6c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:37.330797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61aee57a79b44f328e876d1b04268e6c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:37.332808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:37.379599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61aee57a79b44f328e876d1b04268e6c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.386874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61aee57a79b44f328e876d1b04268e6c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.387501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61aee57a79b44f328e876d1b04268e6c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.388138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61aee57a79b44f328e876d1b04268e6c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.400332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b6ceea59-ab3f-4493-8861-2b4cb7e957c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61aee57a79b44f328e876d1b04268e6c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.412854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1085/2151] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:37 2022] DELETE /identity/v3/users/61aee57a79b44f328e876d1b04268e6c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:37.693049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:37.694487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:37.707953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e09754-48ab-4c81-83c4-632b7981d4e0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:37.755734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:37.777560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:37.778435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:37.778720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:37.778996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] PATH_INFO: `/v3/projects/41ed2da01e134de38060ecf348bc159c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:37.780233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ed2da01e134de38060ecf348bc159c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:37.782161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:37.828211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ed2da01e134de38060ecf348bc159c', 'request_id': 'req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.834658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ed2da01e134de38060ecf348bc159c', 'request_id': 'req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.840298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ed2da01e134de38060ecf348bc159c', 'request_id': 'req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.853145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] The token cache is being invalidate because project 41ed2da01e134de38060ecf348bc159c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:37.853439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.854076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c304c39-87a2-44d4-ae02-101ba5cd8f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:37.855625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1067/2152] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:37 2022] DELETE /identity/v3/projects/41ed2da01e134de38060ecf348bc159c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:37.861114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:37.861576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:37.861687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:37.879833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:37.881059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:37.938380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1086/2153] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:27:37.946055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:37.946349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:37.946649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:37.964600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:37.965795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:37.974849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1068/2154] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:37.978888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:37.989085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a983c613-2a10-439f-ae15-9b04f9d2fce9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:37.993536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:37.994183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:37.994465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:37.994735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:37.995463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:37.995765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:37.997649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.009248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1087/2155] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:27:38.013281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:38.022326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c222-e9ec-4254-a278-24a2a1348256 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:38.027139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:38.027735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.028021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.028330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.029092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:38.031030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.056954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1069/2156] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:38 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:38.060996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:38.071343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb8a21d-3e1a-4b06-a813-d0b59fa17763 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:38.123024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:38.133417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:38.134141 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.134649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.135088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.135861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:38.138005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.157656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1088/2157] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:38 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:38.162584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:38.172762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e13bd4-608b-41fd-964a-c4aab4c4cca7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:38.177385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:38.178043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.178324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.178601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.179327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:38.180954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.192622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1070/2158] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:38.196529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:38.206958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ad66-7286-47a4-977c-44bbf22d3a46 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:38.211570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8426697a-98ab-4f32-943c-68767583a243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:38.212260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426697a-98ab-4f32-943c-68767583a243 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.212545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426697a-98ab-4f32-943c-68767583a243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.212820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8426697a-98ab-4f32-943c-68767583a243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.213546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426697a-98ab-4f32-943c-68767583a243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:38.215132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8426697a-98ab-4f32-943c-68767583a243 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.228931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1089/2159] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:38.230143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:38.241587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5186-af15-4b12-ac2c-0331e878ce95 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:38.246296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:38.247466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.247744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.248016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] PATH_INFO: `/v3/projects/ea10b2a06e5348df8bf4f47e6953bbcd/users/db8156817ef644e4a516d3a0730fca22/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.264189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea10b2a06e5348df8bf4f47e6953bbcd, user_id=db8156817ef644e4a516d3a0730fca22, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:38.266950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.292040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1071/2160] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:38 2022] PUT /identity/v3/projects/ea10b2a06e5348df8bf4f47e6953bbcd/users/db8156817ef644e4a516d3a0730fca22/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:38.294436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.294787 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.295133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.324036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] MFA Rules not processed for user `db8156817ef644e4a516d3a0730fca22`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:38.325288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:38.389816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1090/2161] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:38.401126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:38.402845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:38.458720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79af3b3-1fd0-4e57-b8fb-772ea11648b8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:38.462303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a167e72a-7006-4ce9-af06-9611da7872ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:38.462938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e72a-7006-4ce9-af06-9611da7872ae service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:38.463220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e72a-7006-4ce9-af06-9611da7872ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:38.463491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a167e72a-7006-4ce9-af06-9611da7872ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:38.471583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e72a-7006-4ce9-af06-9611da7872ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:38.473083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a167e72a-7006-4ce9-af06-9611da7872ae service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:38.481683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1072/2162] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:48.386959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=41,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:48.477724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:57.121458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:57.182761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42628a7-8423-4e36-ac81-c7148b164182 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:57.187458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:57.188357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.188640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.189036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] PATH_INFO: `/v3/users/db8156817ef644e4a516d3a0730fca22` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.190253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] RBAC: Authorizing `identity:delete_user(user_id=db8156817ef644e4a516d3a0730fca22)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:57.192335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:57.243939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db8156817ef644e4a516d3a0730fca22'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.251948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db8156817ef644e4a516d3a0730fca22'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.252504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db8156817ef644e4a516d3a0730fca22'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.253322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db8156817ef644e4a516d3a0730fca22'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.267278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db8156817ef644e4a516d3a0730fca22'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.281095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1091/2163] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:27:57 2022] DELETE /identity/v3/users/db8156817ef644e4a516d3a0730fca22 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:57.287142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.287617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.287714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.305133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:57.306331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:57.364339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1073/2164] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:57.374088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:57.375477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:57.433711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb68df-dc39-4c7a-a26f-87bdd393165c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:57.437576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc8ef9-3448-4174-829c-22a2c561a1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:57.438334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc8ef9-3448-4174-829c-22a2c561a1ff service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.438635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc8ef9-3448-4174-829c-22a2c561a1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.438910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc8ef9-3448-4174-829c-22a2c561a1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.448750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc8ef9-3448-4174-829c-22a2c561a1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:57.450553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc8ef9-3448-4174-829c-22a2c561a1ff service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:57.461632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1092/2165] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:27:57.733800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:57.735252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:57.748127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a1b2b-bfa5-4888-a061-6e040160a671 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:57.752869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:57.753734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.754012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.754290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] PATH_INFO: `/v3/projects/ea10b2a06e5348df8bf4f47e6953bbcd` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.755516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea10b2a06e5348df8bf4f47e6953bbcd)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:57.757485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:57.804282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea10b2a06e5348df8bf4f47e6953bbcd', 'request_id': 'req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.810549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea10b2a06e5348df8bf4f47e6953bbcd', 'request_id': 'req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.816129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea10b2a06e5348df8bf4f47e6953bbcd', 'request_id': 'req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.828001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] The token cache is being invalidate because project ea10b2a06e5348df8bf4f47e6953bbcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:27:57.828325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.828944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-65ca0b5c-ba8a-4ef1-b410-d1a1a3aebab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:27:57.830494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1074/2166] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:27:57 2022] DELETE /identity/v3/projects/ea10b2a06e5348df8bf4f47e6953bbcd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:57.836129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.836423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.836728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.854608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:57.855798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:57.872381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:57.914665 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1093/2167] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:57.922490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.922787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.923073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.942351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:57.943552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:57.951454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1075/2168] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:57.955612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:57.965406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741cc72-8ba9-4631-921c-cf754e9340c5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:57.970277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:57.970923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:57.971203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:57.971478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:57.972779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:27:57.973085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:57.974737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:57.985961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1094/2169] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:27:57.990832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.000474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994db51-a3ec-476a-9834-fe3cddd8cebf None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.005241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.005903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.006185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.006463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.007434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.010000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.042221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1076/2170] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:57 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:58.045159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.056437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e5244-038f-4f0e-9583-50afa7344052 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.120394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.121340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.121627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.122164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.122671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.124714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.145874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1095/2171] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:58 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:58.150200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.161410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a7e870-2c40-47c0-9137-a039a48052c8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.166359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.167040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.167326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.167748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.168422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.170171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.184656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1077/2172] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:58.188246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.199339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548fa77d-4c32-49e5-aca2-637444d47277 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.204191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfb5db-d61d-4cfa-b978-ff7aabdb648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.204862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfb5db-d61d-4cfa-b978-ff7aabdb648b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.205150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfb5db-d61d-4cfa-b978-ff7aabdb648b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.205427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfb5db-d61d-4cfa-b978-ff7aabdb648b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.206158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfb5db-d61d-4cfa-b978-ff7aabdb648b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.207863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfb5db-d61d-4cfa-b978-ff7aabdb648b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.219401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1096/2173] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:58.226453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.237390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0b255-88f4-4474-8508-07c17ac8d019 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.242817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.243669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.243958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.244274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] PATH_INFO: `/v3/projects/9ce95dac3fcd4633837426c8370f1875/users/e45ce9bfb0dd497b9c06d5f4a5130573/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.260103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ce95dac3fcd4633837426c8370f1875, user_id=e45ce9bfb0dd497b9c06d5f4a5130573, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.262768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.276871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1078/2174] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:58 2022] PUT /identity/v3/projects/9ce95dac3fcd4633837426c8370f1875/users/e45ce9bfb0dd497b9c06d5f4a5130573/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:27:58.349340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.349633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.349917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.377935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] MFA Rules not processed for user `e45ce9bfb0dd497b9c06d5f4a5130573`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:58.379137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:58.473747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1097/2175] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:58.478473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.502608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:58.548046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3eb0f-6e12-4f3e-83f1-8c1889e9f4ae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.553910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.554581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.554869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.555157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.555867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.558366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.585677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1079/2176] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:58 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:27:58.589798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.601211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ef6df3-4568-4587-99d6-2a5d452a9cdb None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.613554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:58.658671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.659374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.659727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.660014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.660928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.662958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.689434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1098/2177] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:58 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:58.690621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.700735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374a242-b554-4cde-bf00-199ec9fc4040 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.705416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.706074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.706360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.707132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.707896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.709603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.723246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1080/2178] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:58.724431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.734413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f05d96-0a20-4cd5-bda9-0517bc78d373 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.741354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63b927-66d7-4de3-9a21-ffe38ba7c7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.741871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b927-66d7-4de3-9a21-ffe38ba7c7f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.742174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b927-66d7-4de3-9a21-ffe38ba7c7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.742482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63b927-66d7-4de3-9a21-ffe38ba7c7f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.743202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b927-66d7-4de3-9a21-ffe38ba7c7f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.745066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63b927-66d7-4de3-9a21-ffe38ba7c7f5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.756424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1099/2179] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:58.760677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.773524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e5118-965e-4956-bdb5-d7ee30943497 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.776490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.777446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.777723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.778005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] PATH_INFO: `/v3/projects/42a56add6e5941fb8b94ef67a732f36d/users/9b458b1deaf449809d9fcc9498475a9d/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.794466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a56add6e5941fb8b94ef67a732f36d, user_id=9b458b1deaf449809d9fcc9498475a9d, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:58.797192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:58.811820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1081/2180] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:58 2022] PUT /identity/v3/projects/42a56add6e5941fb8b94ef67a732f36d/users/9b458b1deaf449809d9fcc9498475a9d/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:58.822246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.822586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.822885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.852636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] MFA Rules not processed for user `9b458b1deaf449809d9fcc9498475a9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:58.853941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:58.919435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1100/2181] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:58.923766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:58.987310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1615c5-e3bb-4934-893c-f6e6919abe3c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:58.996945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:58.997613 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:58.997901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:58.998181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:58.998895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.000886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.027083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1082/2182] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:27:58 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:59.031102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.042165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b0040d-6765-4de6-babb-f7ee362c604c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.098322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.099018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.099303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.099583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.100297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.102239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.122094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1101/2183] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:27:59 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:27:59.126109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.137140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3758a-f59c-4d92-9cd5-1199c4fe17ae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.142057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.142719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.143004 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.143281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.144051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.145679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.157216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1083/2184] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:59.161295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.171178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9c3e-ae83-4076-89bf-2357a4488f2b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.176220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.177088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.177370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.177646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] PATH_INFO: `/v3/projects/ea41d2b0eebf4e3196f9d6d0c72f6ec3/users/381cfb52e1bf4b4f846af64124596211/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.189644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:59.194313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea41d2b0eebf4e3196f9d6d0c72f6ec3, user_id=381cfb52e1bf4b4f846af64124596211, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.197773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.211306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1102/2185] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:59 2022] PUT /identity/v3/projects/ea41d2b0eebf4e3196f9d6d0c72f6ec3/users/381cfb52e1bf4b4f846af64124596211/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:59.215253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.225618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55810d98-12bc-4c30-b728-3e5ccbc9feca admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.267178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:27:59.283386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f59ea9b-ef31-45f8-aef9-9178ed6fcb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.284054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59ea9b-ef31-45f8-aef9-9178ed6fcb4e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.284370 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59ea9b-ef31-45f8-aef9-9178ed6fcb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.284703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f59ea9b-ef31-45f8-aef9-9178ed6fcb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.285834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59ea9b-ef31-45f8-aef9-9178ed6fcb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.287417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f59ea9b-ef31-45f8-aef9-9178ed6fcb4e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.297056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1084/2186] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:27:59.302764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.314629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309789b4-4cec-4519-9bfa-d763deb6fa1f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.319952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.320867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.321153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.321436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] PATH_INFO: `/v3/projects/ea41d2b0eebf4e3196f9d6d0c72f6ec3/users/381cfb52e1bf4b4f846af64124596211/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.323426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea41d2b0eebf4e3196f9d6d0c72f6ec3, user_id=381cfb52e1bf4b4f846af64124596211, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.326124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.336984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1103/2187] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:27:59 2022] PUT /identity/v3/projects/ea41d2b0eebf4e3196f9d6d0c72f6ec3/users/381cfb52e1bf4b4f846af64124596211/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:27:59.347236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.348258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.348727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.380580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] MFA Rules not processed for user `381cfb52e1bf4b4f846af64124596211`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:27:59.381796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:59.454442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1085/2188] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:27:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:27:59.465943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:27:59.467620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:27:59.546918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e4b7d-09c3-4878-9444-0315150193c8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:27:59.551435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:27:59.552118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:27:59.552397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:27:59.552670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:27:59.620578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:27:59.622386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:27:59.633572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1104/2189] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:27:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:01.116211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:01.126125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d482c65-2785-453f-97d0-163e8cc7999a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:01.130332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:01.130977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:01.131265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:01.131546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:01.188960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:01.190411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:01.199861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1086/2190] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:01.755594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:01.764033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038b4c0-cb96-43f6-bd67-8415507d1717 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:01.767595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3ce69-b2b4-451f-b0f4-ec0dcdad9dd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:01.768284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3ce69-b2b4-451f-b0f4-ec0dcdad9dd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:01.768565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3ce69-b2b4-451f-b0f4-ec0dcdad9dd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:01.768853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3ce69-b2b4-451f-b0f4-ec0dcdad9dd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:01.781610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3ce69-b2b4-451f-b0f4-ec0dcdad9dd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:01.783043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e3ce69-b2b4-451f-b0f4-ec0dcdad9dd5 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:01.793085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1105/2191] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:01 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:05.672204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:05.744993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c5372-3746-439a-8fbe-efb81d646823 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:05.750351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:05.751264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:05.751545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:05.751822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] PATH_INFO: `/v3/users/e45ce9bfb0dd497b9c06d5f4a5130573` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:05.753577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e45ce9bfb0dd497b9c06d5f4a5130573)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:05.755545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:05.760765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:05.806041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e45ce9bfb0dd497b9c06d5f4a5130573'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:05.814160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e45ce9bfb0dd497b9c06d5f4a5130573'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:05.814755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e45ce9bfb0dd497b9c06d5f4a5130573'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:05.815381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e45ce9bfb0dd497b9c06d5f4a5130573'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:05.823943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e45ce9bfb0dd497b9c06d5f4a5130573'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:05.841635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1087/2192] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:28:05 2022] DELETE /identity/v3/users/e45ce9bfb0dd497b9c06d5f4a5130573 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:05.846235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa071c3-5915-4cc6-a98d-cfbd59c6f1a8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:05.846684 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa071c3-5915-4cc6-a98d-cfbd59c6f1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:05.846824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa071c3-5915-4cc6-a98d-cfbd59c6f1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:05.867986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4fa071c3-5915-4cc6-a98d-cfbd59c6f1a8 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:05.869364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4fa071c3-5915-4cc6-a98d-cfbd59c6f1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:05.931524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1106/2193] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:05.941524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:05.943149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:06.005040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a89f19-8cee-4574-9efa-ed611a46b592 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:06.008703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:06.009619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:06.009619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:06.009901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:06.020307 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:06.022067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:06.031709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1088/2194] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:06.092874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:06.093163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:06.093441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:06.111066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] MFA Rules not processed for user `025ea412ccfa455aa84d051244902cd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:06.112298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:06.125771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:06.172376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1107/2195] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:06.316893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:06.318365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:06.338517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3128fd-82f7-4aaa-beb9-24d1a612d981 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:06.343515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:06.344424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:06.344701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:06.345034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] PATH_INFO: `/v3/projects/9ce95dac3fcd4633837426c8370f1875` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:06.346224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ce95dac3fcd4633837426c8370f1875)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:06.350946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:06.401091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ce95dac3fcd4633837426c8370f1875', 'request_id': 'req-0a3cba93-3399-4794-abb3-9b5d8461a080'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.407138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ce95dac3fcd4633837426c8370f1875', 'request_id': 'req-0a3cba93-3399-4794-abb3-9b5d8461a080'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.413464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ce95dac3fcd4633837426c8370f1875', 'request_id': 'req-0a3cba93-3399-4794-abb3-9b5d8461a080'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.427882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] The token cache is being invalidate because project 9ce95dac3fcd4633837426c8370f1875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:28:06.428323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.429352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.431347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1089/2196] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:28:06 2022] DELETE /identity/v3/projects/9ce95dac3fcd4633837426c8370f1875 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:06.435061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:06.446598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace01b4-b655-4f0a-950d-743224e905f1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:06.509201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:06.510172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:06.510532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:06.510881 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] PATH_INFO: `/v3/users/9b458b1deaf449809d9fcc9498475a9d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:06.512329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b458b1deaf449809d9fcc9498475a9d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:06.514352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:06.586090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b458b1deaf449809d9fcc9498475a9d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.594687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b458b1deaf449809d9fcc9498475a9d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.595324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b458b1deaf449809d9fcc9498475a9d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.595948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b458b1deaf449809d9fcc9498475a9d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.611027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b458b1deaf449809d9fcc9498475a9d'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:06.627027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1108/2197] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:28:06 2022] DELETE /identity/v3/users/9b458b1deaf449809d9fcc9498475a9d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:06.958381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:06.960859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:06.981240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cba93-3399-4794-abb3-9b5d8461a080 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:06.990122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:07.075961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:07.077177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:07.077628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:07.078046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] PATH_INFO: `/v3/projects/42a56add6e5941fb8b94ef67a732f36d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:07.080010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a56add6e5941fb8b94ef67a732f36d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:07.085323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:07.173895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a56add6e5941fb8b94ef67a732f36d', 'request_id': 'req-7391a881-fe25-4ebf-9632-588dfcbd13dd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.181759 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a56add6e5941fb8b94ef67a732f36d', 'request_id': 'req-7391a881-fe25-4ebf-9632-588dfcbd13dd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.188846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a56add6e5941fb8b94ef67a732f36d', 'request_id': 'req-7391a881-fe25-4ebf-9632-588dfcbd13dd'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.204667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] The token cache is being invalidate because project 42a56add6e5941fb8b94ef67a732f36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:28:07.205152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.207202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.209010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1090/2198] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:28:06 2022] DELETE /identity/v3/projects/42a56add6e5941fb8b94ef67a732f36d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:07.213718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:07.216805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:07.232006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db2ad8-8513-4f31-a0f1-23f22f3ac702 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:07.282919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:07.302601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:07.303513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:07.303794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:07.304065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] PATH_INFO: `/v3/users/381cfb52e1bf4b4f846af64124596211` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:07.305390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] RBAC: Authorizing `identity:delete_user(user_id=381cfb52e1bf4b4f846af64124596211)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:07.307340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:07.311508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:07.313857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:07.373471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '381cfb52e1bf4b4f846af64124596211'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.383986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '381cfb52e1bf4b4f846af64124596211'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.384782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '381cfb52e1bf4b4f846af64124596211'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.385552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '381cfb52e1bf4b4f846af64124596211'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.401449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '381cfb52e1bf4b4f846af64124596211'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.404744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7391a881-fe25-4ebf-9632-588dfcbd13dd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:07.410456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:07.411193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:07.411562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:07.411844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:07.413718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:07.417805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1109/2199] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:28:07 2022] DELETE /identity/v3/users/381cfb52e1bf4b4f846af64124596211 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:28:07.431379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:07.433210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cac0c5-99d8-4500-b7d7-696a78b6dee2 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:07.446243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1091/2200] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:28:07.710743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:07.712381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:07.792261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef770e-7010-474c-a574-33d0c521343f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:07.796897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:07.797758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:07.798038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:07.798314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] PATH_INFO: `/v3/projects/ea41d2b0eebf4e3196f9d6d0c72f6ec3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:07.799485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea41d2b0eebf4e3196f9d6d0c72f6ec3)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:07.802218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:07.850803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea41d2b0eebf4e3196f9d6d0c72f6ec3', 'request_id': 'req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.860346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea41d2b0eebf4e3196f9d6d0c72f6ec3', 'request_id': 'req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.867457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea41d2b0eebf4e3196f9d6d0c72f6ec3', 'request_id': 'req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.871510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:07.879972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] The token cache is being invalidate because project ea41d2b0eebf4e3196f9d6d0c72f6ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:28:07.880237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.881115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-054d74e1-d38e-4f7f-92d2-cdfcd10b5a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:07.882801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1110/2201] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:28:07 2022] DELETE /identity/v3/projects/ea41d2b0eebf4e3196f9d6d0c72f6ec3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:07.887635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:07.887977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:07.888287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:07.909590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:07.910972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:07.935541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:07.976751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1092/2202] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:07.984631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:07.984915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:07.985254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.006065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:08.007626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:08.020016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1111/2203] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:08.021433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.033123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf12f17-26ae-4eea-ab4a-cc021b33e7dc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.038055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.038757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.039038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.039316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.040051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:28:08.040384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.042011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.058399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1093/2204] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 19:28:08.059618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.070806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffb97a-01eb-4bb9-ad84-a86d580dbb7b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.075807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.076342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.076630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.076990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.078511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.079614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.106689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1112/2205] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:28:08.111009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.123513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad93e15-d016-4db4-8d5c-7d2f71c90456 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.183662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.184282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.184565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.184840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.185527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.187433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.208684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1094/2206] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:28:08 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:28:08.212968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.224262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d73963-c529-45d8-bdc7-515d528d5aa1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.228948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.229607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.229889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.230159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.230888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.232657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.245653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1113/2207] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:28:08.250097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.260940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60bd161-785c-4634-b5fd-928117741db9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.265628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bce62c-a438-47b6-888c-3ba80fedac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.266294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bce62c-a438-47b6-888c-3ba80fedac39 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.266584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bce62c-a438-47b6-888c-3ba80fedac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.266865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bce62c-a438-47b6-888c-3ba80fedac39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.267632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bce62c-a438-47b6-888c-3ba80fedac39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.270110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bce62c-a438-47b6-888c-3ba80fedac39 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.283013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1095/2208] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:28:08.285421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.295361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ff3c4-099a-49dc-a18e-7d6a0b6fe124 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.299983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.300887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.301168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.301443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] PATH_INFO: `/v3/projects/6df8928125c74f3bb0f50062b24b535e/users/a07b2d74104243f3b3b39db918da1fd6/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.315443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df8928125c74f3bb0f50062b24b535e, user_id=a07b2d74104243f3b3b39db918da1fd6, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.318127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.330512 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1114/2209] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:28:08 2022] PUT /identity/v3/projects/6df8928125c74f3bb0f50062b24b535e/users/a07b2d74104243f3b3b39db918da1fd6/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:08.341796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e032d9-a9c6-43c2-80e2-30a87de9a83f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.342087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e032d9-a9c6-43c2-80e2-30a87de9a83f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.342371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e032d9-a9c6-43c2-80e2-30a87de9a83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.368892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f5e032d9-a9c6-43c2-80e2-30a87de9a83f None None] MFA Rules not processed for user `a07b2d74104243f3b3b39db918da1fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:08.370079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f5e032d9-a9c6-43c2-80e2-30a87de9a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:08.438726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1096/2210] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:08.450054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:08.451912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:08.515771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f05ee-fff2-495b-b8aa-e141dce6adcc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:08.520927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703a989c-1b61-4b97-b132-966d5b311c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:08.521926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a989c-1b61-4b97-b132-966d5b311c06 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:08.523251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a989c-1b61-4b97-b132-966d5b311c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:08.523614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703a989c-1b61-4b97-b132-966d5b311c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:08.534045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a989c-1b61-4b97-b132-966d5b311c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:08.536375 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703a989c-1b61-4b97-b132-966d5b311c06 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:08.547968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1115/2211] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:09.922694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3ec8a-73b1-4d26-8bf2-f199dfa9c40a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:09.923345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3ec8a-73b1-4d26-8bf2-f199dfa9c40a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:09.923345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3ec8a-73b1-4d26-8bf2-f199dfa9c40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:09.945118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e6b3ec8a-73b1-4d26-8bf2-f199dfa9c40a None None] MFA Rules not processed for user `a07b2d74104243f3b3b39db918da1fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:09.946584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e6b3ec8a-73b1-4d26-8bf2-f199dfa9c40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:09.954777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1097/2212] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:18.542342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:19.936910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:20.779478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:20.779976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:20.780076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:20.799882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] MFA Rules not processed for user `918edf180b5b40f58994469fb37d25c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:20.801140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:20.867995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1116/2213] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:30.859664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=25 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:43.135952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307104e2-6075-4dc4-a2e8-0cb2090d2bb4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:43.136423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307104e2-6075-4dc4-a2e8-0cb2090d2bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:43.136554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307104e2-6075-4dc4-a2e8-0cb2090d2bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:43.154995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-307104e2-6075-4dc4-a2e8-0cb2090d2bb4 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:43.156262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-307104e2-6075-4dc4-a2e8-0cb2090d2bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:43.217066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1098/2214] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:28:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:43.262988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:43.264487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:43.337146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe0c7af-3c8e-497b-9776-42d836fe087a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:43.342077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:43.342724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:43.343010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:43.343291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:43.351625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:43.353257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:43.362146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1117/2215] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:28:43.875261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc42de-831a-4312-8963-8af2c7b64a0a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:43.875556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc42de-831a-4312-8963-8af2c7b64a0a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:43.875838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc42de-831a-4312-8963-8af2c7b64a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:43.894991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5adc42de-831a-4312-8963-8af2c7b64a0a None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:43.896207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5adc42de-831a-4312-8963-8af2c7b64a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:43.903603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1099/2216] 10.222.0.17 () {58 vars in 967 bytes} [Fri Jul 15 19:28:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:43.915976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:43.926298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc4710-fc95-4df0-91c5-1e971f3f4ec7 service swift] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:43.931239 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:43.931868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:43.932183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:43.932458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:43.940885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:43.942459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:43.950282 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1118/2217] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:28:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:28:44.161443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cbeee-bd0a-4026-b12c-ecbfd14bfd94 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:44.161947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cbeee-bd0a-4026-b12c-ecbfd14bfd94 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:44.162040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04cbeee-bd0a-4026-b12c-ecbfd14bfd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:44.179845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b04cbeee-bd0a-4026-b12c-ecbfd14bfd94 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:44.181094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b04cbeee-bd0a-4026-b12c-ecbfd14bfd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:44.188349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1100/2218] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:28:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:44.200647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:44.211572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4f399-1430-4a79-81d4-04d293045303 service swift] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:44.216527 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:44.217161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:44.217443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:44.217727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:44.226290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:44.227869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:44.236509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1119/2219] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:28:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:28:44.639630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:44.639987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:44.640367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:44.659007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:44.660262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:44.669437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1101/2220] 10.222.0.17 () {58 vars in 967 bytes} [Fri Jul 15 19:28:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:28:44.683601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:44.695867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c6441-4b95-43d3-b95e-0360055e2289 service swift] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:44.701912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20733c0c-b272-4699-9479-bb68c8af0262 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:44.702654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20733c0c-b272-4699-9479-bb68c8af0262 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:44.703029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20733c0c-b272-4699-9479-bb68c8af0262 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:44.703698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20733c0c-b272-4699-9479-bb68c8af0262 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:44.724322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20733c0c-b272-4699-9479-bb68c8af0262 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:44.726080 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20733c0c-b272-4699-9479-bb68c8af0262 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:44.736993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1120/2221] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:28:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:28:46.916448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:46.981923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331448f-92ca-4508-a678-887b3d25ad82 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:46.986624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:46.987511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:46.987791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:46.988110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] PATH_INFO: `/v3/users/a07b2d74104243f3b3b39db918da1fd6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:46.989316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07b2d74104243f3b3b39db918da1fd6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:46.991281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:47.039452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07b2d74104243f3b3b39db918da1fd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.047400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07b2d74104243f3b3b39db918da1fd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.048025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07b2d74104243f3b3b39db918da1fd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.048650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07b2d74104243f3b3b39db918da1fd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.056372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07b2d74104243f3b3b39db918da1fd6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.073799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1102/2222] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:28:46 2022] DELETE /identity/v3/users/a07b2d74104243f3b3b39db918da1fd6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:47.078267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.078749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.078854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.096580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:28:47.097991 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:47.156523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1121/2223] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:28:47.166714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:47.168223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:47.175005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:47.232877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7430f054-fc3f-47ee-85ed-66915a4a356a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:47.236564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:47.237215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.237503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.237783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.247188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:47.248982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:47.259147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1103/2224] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:28:47.429355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:28:47.430760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:28:47.444484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97090f8-4ae5-47c0-801d-36479878ea26 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:28:47.449046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:28:47.449908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:28:47.450188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:28:47.450465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] PATH_INFO: `/v3/projects/6df8928125c74f3bb0f50062b24b535e` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:28:47.451644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df8928125c74f3bb0f50062b24b535e)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:28:47.453592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:28:47.502028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:47.506143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df8928125c74f3bb0f50062b24b535e', 'request_id': 'req-dee4021c-f2cc-4eb2-87e4-84033352d8d2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.514576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df8928125c74f3bb0f50062b24b535e', 'request_id': 'req-dee4021c-f2cc-4eb2-87e4-84033352d8d2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.521688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df8928125c74f3bb0f50062b24b535e', 'request_id': 'req-dee4021c-f2cc-4eb2-87e4-84033352d8d2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.531943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] The token cache is being invalidate because project 6df8928125c74f3bb0f50062b24b535e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:28:47.532259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.532895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:28:47.534970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1122/2225] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:28:47 2022] DELETE /identity/v3/projects/6df8928125c74f3bb0f50062b24b535e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:28:57.254443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=24 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:28:57.528922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:29:55.698410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:55.707577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602dec36-61a7-47b5-8408-68410ccdf14b service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:55.760476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:55.761127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:55.761416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:55.761695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:55.763486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:55.836878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:55.838442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31304a-d9ba-4875-a003-476b051e2f11 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:55.849923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1104/2226] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:29:59.326483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:29:59.327798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:59.340392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee4021c-f2cc-4eb2-87e4-84033352d8d2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:29:59.343917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:29:59.344589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:29:59.344875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:29:59.345154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:29:59.347482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:29:59.360507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:29:59.362012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:29:59.373037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1123/2227] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:29:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:05.578960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:05.579433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:05.579668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:05.598510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:30:05.599716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:05.656964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1105/2228] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:30:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:30:05.703993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:05.706046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:05.775843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15cf4e-931f-4a5f-b127-633773e28887 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:05.780889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:05.781531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:05.781812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:05.782085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:05.790297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:05.791878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:05.800416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1124/2229] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:30:08.542635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:08.544778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:08.561967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872516f-5d3a-4192-8130-f973b69f0e8c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:08.574947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:08.576885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:08.576885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:08.582800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:08.586617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:08.643775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:08.662918 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:08.664692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:08.675555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1106/2230] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:11.438467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:11.441077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:11.461471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6977e-2a5e-4020-980c-9a4bd9d43edc service swift] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:11.468261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:11.468912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:11.469199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:11.469474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:11.471487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:11.537921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:11.539550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:11.553640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1125/2231] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:30:18.670379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:18.824723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:18.826309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:18.888486 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5db43-37da-4070-a37c-6378eda5f7d7 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:18.892198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:18.892964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:18.893246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:18.893522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:18.895336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:18.955471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:18.956966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b5d1b-15ef-4016-95d2-72745839e3fe service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:18.966002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1107/2232] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:19.431572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:19.432988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:19.445188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97523ae-8513-4185-bd7f-1e2c4d40dab6 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:19.448851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:19.449488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.449768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.450039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.451918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:19.462044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:19.463491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e8e2b-aa05-4016-918c-83ee93c6ac67 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:19.472241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1126/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:19.579357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e159c-341c-4c2c-9c6d-8a126875063c None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.580420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e159c-341c-4c2c-9c6d-8a126875063c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.580770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998e159c-341c-4c2c-9c6d-8a126875063c None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.582584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1108/2234] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:30:19.590077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be38fa-7507-4402-9115-d11ebea24fd5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.590452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be38fa-7507-4402-9115-d11ebea24fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.590795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81be38fa-7507-4402-9115-d11ebea24fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.611623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-81be38fa-7507-4402-9115-d11ebea24fd5 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:30:19.612890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-81be38fa-7507-4402-9115-d11ebea24fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:19.623640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1127/2235] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:30:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:30:19.637235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e159c-341c-4c2c-9c6d-8a126875063c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:19.639257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-998e159c-341c-4c2c-9c6d-8a126875063c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:19.728997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998e159c-341c-4c2c-9c6d-8a126875063c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:19.733435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46351cd9-b281-46c1-bc8d-56f1ca879a47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:19.734088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351cd9-b281-46c1-bc8d-56f1ca879a47 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.734372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351cd9-b281-46c1-bc8d-56f1ca879a47 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.734647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46351cd9-b281-46c1-bc8d-56f1ca879a47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.746589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46351cd9-b281-46c1-bc8d-56f1ca879a47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:19.748034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46351cd9-b281-46c1-bc8d-56f1ca879a47 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:19.759939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1109/2236] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:19.841169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a8f4c-1ea2-4ca6-82c7-2bb25464367d None None] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.841467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a8f4c-1ea2-4ca6-82c7-2bb25464367d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.841779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0a8f4c-1ea2-4ca6-82c7-2bb25464367d None None] PATH_INFO: `/` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.843380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1128/2237] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:30:19.848842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c78c8b-4a9e-41e1-8905-f8de82ee7f4b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.849126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c78c8b-4a9e-41e1-8905-f8de82ee7f4b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.849405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c78c8b-4a9e-41e1-8905-f8de82ee7f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.869483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-90c78c8b-4a9e-41e1-8905-f8de82ee7f4b None None] MFA Rules not processed for user `9f135ce75fbc4b12977cc8f009083ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:30:19.870739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-90c78c8b-4a9e-41e1-8905-f8de82ee7f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:19.879321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1110/2238] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:30:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:30:19.891225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a8f4c-1ea2-4ca6-82c7-2bb25464367d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:19.903619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0a8f4c-1ea2-4ca6-82c7-2bb25464367d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:19.907551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c188bef3-eb40-4e64-a80d-897a07efe798 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:19.908491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188bef3-eb40-4e64-a80d-897a07efe798 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:19.908780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188bef3-eb40-4e64-a80d-897a07efe798 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:19.909058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c188bef3-eb40-4e64-a80d-897a07efe798 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:19.917360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188bef3-eb40-4e64-a80d-897a07efe798 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:19.918921 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c188bef3-eb40-4e64-a80d-897a07efe798 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:19.927241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1129/2239] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:20.010762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee8b3-c36f-479e-8743-c49808be246d None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:20.011057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee8b3-c36f-479e-8743-c49808be246d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:20.011338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbee8b3-c36f-479e-8743-c49808be246d None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:20.012772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1111/2240] 10.222.0.17 () {54 vars in 921 bytes} [Fri Jul 15 19:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 15 19:30:20.018502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:20.018960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:20.019319 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:20.039987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] MFA Rules not processed for user `37f8217df17a44a989ce1c7ab8a9ac1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:30:20.041261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:20.051568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1130/2241] 10.222.0.17 () {58 vars in 1018 bytes} [Fri Jul 15 19:30:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:30:20.065645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee8b3-c36f-479e-8743-c49808be246d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:20.082707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbee8b3-c36f-479e-8743-c49808be246d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:20.088637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:20.089297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:20.089580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:20.089852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:20.119123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:20.120681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:20.134519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1112/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:20.209884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:20.211301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:20.256525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:20.290378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4923a-5240-49c4-9760-22ef6d638077 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:20.295477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:20.296173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:20.296461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:20.296741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:20.298733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:20.311797 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:20.314090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:20.326079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1131/2243] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:21.962738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:30:21.972799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec14e24-d1d2-4d77-a1c5-34e36a2892b2 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:30:21.976456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:30:21.977117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:30:21.977405 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:30:21.977684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:30:21.979489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:30:21.994961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:22.046737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:30:22.048406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:30:22.059302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1113/2244] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:30:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:30:30.320713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=16 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:30:32.054378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=20 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:02.595518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:02.605370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8677324-fa6f-4b56-85aa-a5c637f116e2 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:02.608897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:02.609528 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:02.609807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:02.610080 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:02.611849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:02.621468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:02.622889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:02.630828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1132/2245] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:31:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:31:12.627175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:31:20.075947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:31:20.085371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01357c-154b-40be-a057-e425e6ee9cd0 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:31:20.089028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:31:20.089658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:31:20.089942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:31:20.090219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:31:20.092026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:31:20.149842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:31:20.151282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:31:20.161368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1114/2246] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:31:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:31:30.156176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=28 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:34:04.898770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:34:04.958660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a3c8c-789f-4509-a986-f081813b7b43 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:34:04.963662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:34:04.964337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:34:04.964623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:34:04.964902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:34:04.966679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:34:05.001030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:34:05.002628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:34:05.012392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1133/2247] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:34:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:34:15.008194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=36 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:34:56.328038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:34:56.356605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31266a-af75-41e9-87f2-d7fe3d598a3d service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:34:56.361138 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:34:56.361470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:34:56.361750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:34:56.362024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:34:56.379408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:34:56.380877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:34:56.403246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1115/2248] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:34:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:34:58.787005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:34:58.796439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7945fe0a-6dfc-4043-a076-11612ecda3b4 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:34:58.801167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bbc38a-7b88-4d22-92ec-b79b815db744 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:34:58.801818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc38a-7b88-4d22-92ec-b79b815db744 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:34:58.802105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc38a-7b88-4d22-92ec-b79b815db744 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:34:58.802382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bbc38a-7b88-4d22-92ec-b79b815db744 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:34:58.812965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbc38a-7b88-4d22-92ec-b79b815db744 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:34:58.814683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bbc38a-7b88-4d22-92ec-b79b815db744 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:34:58.827538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1134/2249] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:34:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:34:59.449512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:34:59.462980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c3957-f54d-492a-9e16-e9a5bc9e198d service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:34:59.484790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:34:59.486257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:34:59.486756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:34:59.487220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:34:59.489752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:34:59.521922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:34:59.523577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:34:59.546504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1116/2250] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:34:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:02.847229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:02.847699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:02.847792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:02.887538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] MFA Rules not processed for user `bf32cd62eec04de892c3b27fbdd882a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:35:02.888768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:02.895857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1135/2251] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:35:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:35:08.708867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:08.727808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca49e621-0cfb-459a-8fa9-9443cabcf2be service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:08.735266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:08.735951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:08.736276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:08.736558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:08.745664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:08.747103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:08.775676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1117/2252] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:10.869514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:10.883146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b480448-5ef2-4bba-ad95-2073960c4840 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:10.888612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:10.889261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:10.889548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:10.889889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:10.899350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:10.901100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:10.933818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1136/2253] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:11.104314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:11.118957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d610b-905d-49ed-9c82-a5323322d307 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:11.124372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:11.125025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:11.125302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:11.125575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:11.127369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:35:11.139420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:11.141057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:11.152221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1118/2254] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:18.837303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:35:18.847532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418e02-f457-43d4-a5e3-e3d3ca6f8d13 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:35:18.851157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d3b8d-01d6-43b1-b37e-c0bce3e32c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:35:18.851799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d3b8d-01d6-43b1-b37e-c0bce3e32c3d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:35:18.852110 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d3b8d-01d6-43b1-b37e-c0bce3e32c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:35:18.852392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d3b8d-01d6-43b1-b37e-c0bce3e32c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:35:18.860919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d3b8d-01d6-43b1-b37e-c0bce3e32c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:35:18.862607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d3b8d-01d6-43b1-b37e-c0bce3e32c3d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:35:18.889628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1137/2255] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:35:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:35:21.147583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=56 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:35:28.886915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=44 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:09.573704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:09.583185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ff470-f62a-4c91-8e40-01c17dde9a4e service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:09.586756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:09.587390 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:09.587700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:09.588021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:09.595176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:09.596649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:09.604467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1119/2256] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:19.600983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:19.744932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.745410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.745562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.804363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:19.805576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:19.822465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1138/2257] 10.222.0.17 () {58 vars in 967 bytes} [Fri Jul 15 19:36:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:19.870338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:19.892779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746023e6-e317-42cf-b516-c65b61c1fbf8 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:19.898336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:19.898993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:19.899263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:19.899544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:19.907665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:19.909287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:19.918378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1120/2258] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 19:36:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 19:36:20.091974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.104843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645e41b-6752-4f9b-bf9d-98afb1b4a7a4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.110415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.111170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.111402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.112005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.122483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.124194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.144972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1139/2259] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:36:20.413563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:20.423064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc56bac-806c-4fb3-a134-6c27761d2c22 service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:20.427577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db9d6eb-fc87-4ca5-8ab7-791e718a52e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:20.428562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9d6eb-fc87-4ca5-8ab7-791e718a52e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:20.428920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9d6eb-fc87-4ca5-8ab7-791e718a52e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:20.429273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db9d6eb-fc87-4ca5-8ab7-791e718a52e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:20.439753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9d6eb-fc87-4ca5-8ab7-791e718a52e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:20.443871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db9d6eb-fc87-4ca5-8ab7-791e718a52e4 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:20.452969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1121/2260] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:21.380865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:21.393230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff1ef99-56e8-4d3d-b85c-4427b85554c1 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:21.400418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:21.401244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:21.401646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:21.401876 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:21.404114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:21.417386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:21.419099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:21.419453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345ca81-46ab-49ec-80dc-9bfd537184c0 None None] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:21.419909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345ca81-46ab-49ec-80dc-9bfd537184c0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:21.420011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345ca81-46ab-49ec-80dc-9bfd537184c0 None None] PATH_INFO: `/` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:21.421396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1122/2261] 10.222.0.17 () {54 vars in 912 bytes} [Fri Jul 15 19:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 15 19:36:21.426668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c0f4f-dd1d-488b-84a2-cddbddff4292 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:21.427087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c0f4f-dd1d-488b-84a2-cddbddff4292 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:21.427464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c0f4f-dd1d-488b-84a2-cddbddff4292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:21.431134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1140/2262] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:21.494062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-291c0f4f-dd1d-488b-84a2-cddbddff4292 None None] MFA Rules not processed for user `350d4be3651647829ae3f7e903326891`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:21.495548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-291c0f4f-dd1d-488b-84a2-cddbddff4292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:21.504076 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1123/2263] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 19:36:21.528560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:21.541396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc94bcc-268e-4d7a-9987-a836ef51970b service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:21.548491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:21.550177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:21.550517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:21.550801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:21.562987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:21.564681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:21.578506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1141/2264] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:23.901093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:23.901642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:23.901935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:23.945376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] MFA Rules not processed for user `01120eedaa844f24a66024df8d3b2271`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:36:23.946497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:23.953457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1124/2265] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:36:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:36:24.258628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:24.267824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785d2026-0dd3-4c5a-87a9-d06b79319475 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:24.271494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:24.272167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.272455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.272731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.279897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:24.281362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:24.289996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1142/2266] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:24.617695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:36:24.619215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:36:24.634571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2122e2-68a9-4178-a9b9-97403e06cada None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:36:24.639557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:36:24.640302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:36:24.640674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:36:24.640958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:36:24.650042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:36:24.651694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:36:24.662437 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1125/2267] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:36:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:36:34.285683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=50 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:36:34.657643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=46 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:39:07.849683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:39:07.863050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c16095-9faa-4d38-8440-d2be8409f62f service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:39:07.868100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:39:07.869401 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:39:07.869909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:39:07.870193 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:39:07.878959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:39:07.880643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:39:07.892129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1143/2268] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:39:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:39:17.887911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:09.040278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:09.042753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:09.122197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dafbb3-e24f-4ebf-8832-e66fdd4fde41 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:09.128002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:09.128685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:09.128970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:09.129254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:09.131148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:09.205359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:09.206949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:09.216817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1126/2269] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:11.272414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:11.273831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:11.298926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba5721f-51a4-435b-b3ff-d97e50c9378f service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:11.302463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:11.303099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:11.303380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:11.303653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:11.312531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:11.313954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:11.324272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1144/2270] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:18.585457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:18.595256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c2c9f-3970-4c4e-8b4b-ed5893763f93 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:18.598812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:18.599449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:18.599730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:18.600003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:18.608399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:18.609847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:18.618626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1127/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:40:21.319027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=11 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:28.614531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=67 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:40:58.786147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:40:58.787704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:58.857796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526bd14-c9d9-4461-9ecf-06bc025ecff5 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:40:58.861663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:40:58.862472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:40:58.862750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:40:58.862909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:40:58.864862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:40:58.882574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:40:58.884176 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:40:58.896582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1145/2272] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:40:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:41:08.891361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=31 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:13.544398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:13.546140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:13.558457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0419b7c0-38dc-4f63-a735-c5aa05b26f09 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:13.563168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:13.563834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:13.564149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:13.564426 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:13.566376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:13.578018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:13.579662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:13.588037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1128/2273] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:41:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:41:19.844500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:19.863803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:41:19.876963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa7797c-e2ef-40d8-af14-db98f90a2774 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:19.882374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:19.883018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:19.883327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:19.883606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:19.892413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:19.894003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:19.900136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7588-ae96-4f36-8d76-cb6a0ea278c0 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:41:19.903648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:41:19.904347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:41:19.904628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:41:19.904901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:41:19.907470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:41:19.921743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1129/2274] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:41:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:41:19.966049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:41:19.967539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:41:19.979325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1146/2275] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:41:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:41:29.919128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=19 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:41:29.973124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=48 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:42:22.158781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:22.168015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc5b328-251c-4264-b6df-2397279885a2 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:22.171703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:22.172878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:22.173196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:22.173475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:22.175365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:22.190194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:22.191646 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92619189-04e8-478f-af29-7b03df03c7fd service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:22.203173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1130/2276] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:22.835463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:22.838563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:22.856121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc302c-9fed-47f0-b2be-34664b1f0a4d service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:22.862801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:22.863581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:22.864444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:22.864822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:22.876722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:22.879047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:22.900042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1147/2277] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:25.767564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:25.768057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:25.768189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:25.795986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] MFA Rules not processed for user `bf32cd62eec04de892c3b27fbdd882a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:42:25.797427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:25.805189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1131/2278] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:42:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:42:26.111862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:26.120962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc1776-980d-46c4-abb7-f5ec55f11b39 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:26.124533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:26.125733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:26.126024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:26.126301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:26.128477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:26.137750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:26.139177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:26.147542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1148/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:26.458585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:26.460206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:26.532954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad03186d-662f-4734-90b2-dd55a20b9ffa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:26.537887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:26.538536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:26.538817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:26.539099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:26.540987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:42:26.553123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:26.554782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:26.565358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1132/2280] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:34.983057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:42:34.996980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3941b-1865-497c-a54b-96b18541774e service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:42:35.004034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:42:35.004917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:42:35.005203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:42:35.005603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:42:35.015497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:42:35.017148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:42:35.027801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1149/2281] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:42:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:42:36.561157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=44 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:42:45.023273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=18 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:44:15.854077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:44:15.903231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f286dc-cf14-4e1c-8f43-f507af29a3f3 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:44:15.908992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:44:15.909761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:44:15.910046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:44:15.910324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:44:15.912234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:44:15.945406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:44:15.946987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:44:15.957666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1133/2282] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:44:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:44:25.952483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=31 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:09.150072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:09.181045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ea32e7-f717-4e76-943d-83fd36b421d7 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:09.184821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:09.185474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:09.185762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:09.186039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:09.196378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:09.197870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:09.227473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1150/2283] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:11.601012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:11.619444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b2c174-1578-46a1-bb28-987bcd519305 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:11.622958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:11.623580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:11.623861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:11.624166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:11.626011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:11.693542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:11.694973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:11.713503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1134/2284] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:11 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:19.224791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=22 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:19.804987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:45:19.813368 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e84b34-5dea-4a55-b5cc-a94788ced7d8 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:45:19.816983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:45:19.817663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:45:19.817936 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:45:19.818261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:45:19.820056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:45:19.900017 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:45:19.901489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:45:19.919963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1151/2285] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:45:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:45:21.710838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=41 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:45:29.917297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=41 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:46:00.404351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:46:00.423932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d021c2-5f80-48e8-b628-05690cf6a8a7 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:46:00.427545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:46:00.428227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:46:00.428509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:46:00.428784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:46:00.447676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:46:00.449144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:46:00.470081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1135/2286] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:46:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:46:10.467258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=17 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:46:20.236160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:46:20.245963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c763e5-4138-41be-98d9-a490685033c4 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:46:20.249505 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:46:20.250147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:46:20.250432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:46:20.250710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:46:20.258985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:46:20.260463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:46:20.268317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1152/2287] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:46:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:46:20.891248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:46:20.902464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1e05c7-8742-4f1c-bb50-7b5038a8738e service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:46:20.908904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:46:20.909698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:46:20.910107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:46:20.910847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:46:20.921849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:46:20.994611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:46:20.996231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:46:21.005581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1136/2288] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:46:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:46:21.061566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:46:21.063143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:46:21.074689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3420cd3b-5e73-4755-b9cd-99bf98429685 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:46:21.078172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:46:21.078816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:46:21.079097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:46:21.079370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:46:21.088301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:46:21.089776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:46:21.118274 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1153/2289] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:46:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:46:31.000994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=33 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:46:31.114630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=19 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:48:07.718789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:07.728944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3252a64d-4952-4724-a61e-7ed43e0277df service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:07.732702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:07.733347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:07.733631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:07.734178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:07.745158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:07.746601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:07.759150 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1137/2290] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:48:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:48:07.931047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:07.947317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66ac16-aca5-41cc-9b2c-9dbea674bc33 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:07.954315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:07.955200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:07.955599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:07.955978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:07.958896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:07.994925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:08.001051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:08.015681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1154/2291] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:48:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:48:11.493757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:11.503034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907e0a31-936f-46c1-900b-6797fd397864 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:11.506850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:11.507484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:11.507764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:11.508038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:11.509809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:11.597321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:11.598023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:11.621276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1138/2292] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:48:14.900155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:14.936436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5e6a7-f704-4d7e-adb5-8201213b97fb service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:14.941592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:14.942300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:14.942580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:14.942867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:14.945054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:14.958406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:14.960162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:14.970055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:14.972039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:14.973926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1155/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:48:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:48:14.996701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5395e0-d0ef-4f7c-9688-97102f68eb1e service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.002035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.002914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.003196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.003470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] PATH_INFO: `/v3/users/025ea412ccfa455aa84d051244902cd1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.005283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=025ea412ccfa455aa84d051244902cd1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.007210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.057066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '025ea412ccfa455aa84d051244902cd1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.064515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '025ea412ccfa455aa84d051244902cd1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.065140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '025ea412ccfa455aa84d051244902cd1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.065782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '025ea412ccfa455aa84d051244902cd1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.072717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-15aa5c9c-dac6-43c3-8706-8a31bde616b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '025ea412ccfa455aa84d051244902cd1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.090323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1139/2294] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:48:14 2022] DELETE /identity/v3/users/025ea412ccfa455aa84d051244902cd1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:48:15.277987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.289094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da946f1-68d0-48c5-9644-cfe93b681502 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.347374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.348289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.348575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.348854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] PATH_INFO: `/v3/projects/4b566a1f9c8e466585ece82f307f41cb` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.350024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b566a1f9c8e466585ece82f307f41cb)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.351987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.396798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b566a1f9c8e466585ece82f307f41cb', 'request_id': 'req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.403606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b566a1f9c8e466585ece82f307f41cb', 'request_id': 'req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.410196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b566a1f9c8e466585ece82f307f41cb', 'request_id': 'req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.420614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] The token cache is being invalidate because project 4b566a1f9c8e466585ece82f307f41cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:48:15.420902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.421661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d5e5fe6c-c1c7-4c22-a4b1-e1e91d04e9f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:48:15.423519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1156/2295] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:48:15 2022] DELETE /identity/v3/projects/4b566a1f9c8e466585ece82f307f41cb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:48:15.429650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.429936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.430215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.462929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:48:15.488666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:48:15.489809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:15.548488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1140/2296] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:48:15.556194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.556513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.556824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.575504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:48:15.577114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:15.584925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1157/2297] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:48:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:48:15.588986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.599209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f197bfe9-cec2-4184-b57a-df6c26948e18 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.604489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.605147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.605430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.605705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.606433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:48:15.606738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.609116 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.620723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1141/2298] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:48:15.625570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.634850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a27bb1-2816-4cc1-881b-a2ca9c942bd3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.639507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.640184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.640467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.640741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.641420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.643354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.678174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1158/2299] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:48:15 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:48:15.682309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.692864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae370c-024e-492b-86b3-87dba14cd689 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.749945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.750543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.750833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.751115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.751809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.754066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.773389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1142/2300] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:48:15 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:48:15.777556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.788097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fde817-0462-4049-8850-8a195de3f08b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.792827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.793490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.793906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.794050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.794837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.797208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.809742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1159/2301] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:48:15.812068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.823271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a596c5-246c-4eb3-b4e1-98f2795539af admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.828243 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0f637-bf24-486f-be1a-613a658c0267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.828980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f637-bf24-486f-be1a-613a658c0267 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.829180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f637-bf24-486f-be1a-613a658c0267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.829457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0f637-bf24-486f-be1a-613a658c0267 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.830182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f637-bf24-486f-be1a-613a658c0267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.831753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0f637-bf24-486f-be1a-613a658c0267 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.842371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1143/2302] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:48:15.846636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:15.853127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:48:15.857171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfbd0f-3c5a-4ac5-8faf-4bfca3f6fa94 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:15.864398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:15.865359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.865861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.866337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] PATH_INFO: `/v3/projects/0c0ef1b653a5430d8d1675991d907ac7/users/cc85b5d830dd4430a479c6977dbb45ad/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.882837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c0ef1b653a5430d8d1675991d907ac7, user_id=cc85b5d830dd4430a479c6977dbb45ad, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:15.885655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:15.900662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1160/2303] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:48:15 2022] PUT /identity/v3/projects/0c0ef1b653a5430d8d1675991d907ac7/users/cc85b5d830dd4430a479c6977dbb45ad/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:48:15.912427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:15.912728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:15.913012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:15.942446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] MFA Rules not processed for user `cc85b5d830dd4430a479c6977dbb45ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:48:15.943654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:15.993071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:48:16.011448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1144/2304] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:48:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:48:16.016252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.077768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67029d6-d0e2-411b-9dad-b1f570a7d371 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.082390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.083040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.083326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.083603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.084333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.086256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.111544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1161/2305] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:48:16 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:48:16.115756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.127490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566601fa-7482-409b-80d1-5f4a431ed93c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.182338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.183002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.183282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.183556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.184274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.186749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.206030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1145/2306] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:48:16 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:48:16.210014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.221331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64227f0-26f7-49b5-b8e2-0d429724f017 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.226183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.226836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.227120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.227396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.228152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.229753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.241545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1162/2307] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:48:16.245305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.256500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e65a7-8c1c-4d60-8def-89828688b458 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.264199 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.265108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.265459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.265735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] PATH_INFO: `/v3/projects/6d6ea528e2184e9199cec4b6eadb9722/users/781f4e9ad9ea4825bc952eded5902235/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.281354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6ea528e2184e9199cec4b6eadb9722, user_id=781f4e9ad9ea4825bc952eded5902235, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.284015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.297368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1146/2308] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:48:16 2022] PUT /identity/v3/projects/6d6ea528e2184e9199cec4b6eadb9722/users/781f4e9ad9ea4825bc952eded5902235/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:48:16.301347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.311525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c95d09-2732-4d6c-b141-5a28b2356ddc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.366591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49ac74-dbab-4f00-929e-bd6de2d87f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.367267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49ac74-dbab-4f00-929e-bd6de2d87f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.367525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49ac74-dbab-4f00-929e-bd6de2d87f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.367798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49ac74-dbab-4f00-929e-bd6de2d87f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.368549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49ac74-dbab-4f00-929e-bd6de2d87f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.370137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49ac74-dbab-4f00-929e-bd6de2d87f6b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.379976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1163/2309] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:48:16 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:48:16.384120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.394884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d13934-f0b5-493e-a4d9-bf0d01544fc4 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.399452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.400431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.400713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.400986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] PATH_INFO: `/v3/projects/6d6ea528e2184e9199cec4b6eadb9722/users/781f4e9ad9ea4825bc952eded5902235/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.402922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6ea528e2184e9199cec4b6eadb9722, user_id=781f4e9ad9ea4825bc952eded5902235, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.406181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.416031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1147/2310] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:48:16 2022] PUT /identity/v3/projects/6d6ea528e2184e9199cec4b6eadb9722/users/781f4e9ad9ea4825bc952eded5902235/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 19:48:16.425832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.426126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.426409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.452644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] MFA Rules not processed for user `781f4e9ad9ea4825bc952eded5902235`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:48:16.453850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:16.494805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:48:16.525263 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1164/2311] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:48:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:48:16.536924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.538614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:48:16.596933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8743668f-83e7-4905-8f08-30532af2797e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.600871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.601517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.601797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.602073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.611366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.612825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.622102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1148/2312] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:48:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:48:16.694654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:48:16.710858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3461ec2-5be1-4d7a-9787-ec2addbd5c8f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:48:16.714588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:48:16.715236 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:48:16.715522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:48:16.715802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:48:16.775092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:48:16.776650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:48:16.787454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1165/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:48:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:48:26.617955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=75,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:48:26.782792 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=39 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:49:18.835992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:49:18.837416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:49:18.904476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d69c15-a056-4617-a164-75c646d7a6d7 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:49:18.909459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:49:18.910096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:49:18.910379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:49:18.910652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:49:18.912403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:49:18.923409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:49:18.925071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:49:18.934698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1149/2314] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:49:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:49:28.930642 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=32 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:50:09.169018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:50:09.170418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:50:09.246620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383af68b-f2e1-4405-8d14-a835870bce4d service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:50:09.252060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:50:09.252734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:50:09.253019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:50:09.253560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:50:09.255357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:50:09.323667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:50:09.325305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:50:09.334831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1166/2315] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:50:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:50:19.329989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=61 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:51:00.983174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:51:00.984621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:51:01.051153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fab5a7-771b-4581-a2d7-b781a439925f service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:51:01.054597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:51:01.055309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:51:01.055594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:51:01.055867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:51:01.057650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:51:01.071252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:51:01.072777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:51:01.084194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1150/2316] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:51:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:51:11.078733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=31 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:51:33.406790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:51:33.419596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7ce275-50c4-4d9a-bbdb-6c2a0160c7bf service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:51:33.425652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:51:33.426311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:51:33.427181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:51:33.427181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:51:33.429142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:51:33.521639 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:51:33.523202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:51:33.543785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1167/2317] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:51:33.609730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:51:33.621480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b94712c-c1b7-4eff-94b3-8c67b633dbf8 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:51:33.625257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:51:33.625926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:51:33.626210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:51:33.627007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:51:33.629128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:51:33.706106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:51:33.707472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:51:33.726777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1151/2318] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:51:33.846201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:51:33.854913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b29270-ec6b-4313-81e6-bff118287aa6 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:51:33.858519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:51:33.859163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:51:33.859443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:51:33.859718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:51:33.862086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:51:33.937594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:51:33.939709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:51:33.949783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1168/2319] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:51:43.724034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=36 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:51:43.945138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=66 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:53:07.912515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:53:07.914060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:53:07.926104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4369da5-9e70-4133-ae95-55715045f6f7 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:53:07.930021 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:53:07.930662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:53:07.930945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:53:07.931222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:53:07.932976 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:53:07.945991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:53:07.947412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:53:07.958767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1152/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:53:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:53:17.953583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:53:34.379798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:53:34.388207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9ed01-e531-49f7-bbd2-66a97383aff9 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:53:34.391731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:53:34.392818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:53:34.392968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:53:34.393131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:53:34.401304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:53:34.402723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:53:34.412073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1169/2321] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:53:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:53:44.407940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=6 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:54:21.795416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:54:21.843925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6799b89-f6e1-4f40-bfa6-c8b6c17fb8d0 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:54:21.849718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e3852-5110-4736-ba5c-cb468f8d346d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:54:21.850362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e3852-5110-4736-ba5c-cb468f8d346d service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:54:21.850648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e3852-5110-4736-ba5c-cb468f8d346d service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:54:21.850928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e3852-5110-4736-ba5c-cb468f8d346d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:54:21.859365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e3852-5110-4736-ba5c-cb468f8d346d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:54:21.861920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e3852-5110-4736-ba5c-cb468f8d346d service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:54:21.871606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1153/2322] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:54:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:54:31.867518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=22 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:55:09.138735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:09.165695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f5a5d6-c535-41e7-9f0a-3ba7097f92df service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:09.171328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:09.171976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:09.172298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:09.172579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:09.181597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:09.183197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:09.192164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1170/2323] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:19.187994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=12 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:55:33.528268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:33.528795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:33.528920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:33.573847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] MFA Rules not processed for user `837f13aef58b455e95372c6f106d5807`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:55:33.574975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:33.594687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1154/2324] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:55:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:55:33.910724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:33.931237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6231ed-bd5d-4950-9f39-50893ff82802 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:33.934859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:33.935499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:33.935783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:33.936064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:33.937843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:33.947725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:33.949198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:33.957354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1171/2325] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:34.291797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:34.306245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be3ece-eec3-4f82-ab64-98043c9ee0ca None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:34.312591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:34.313563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:34.313952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:34.314433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:34.316521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:34.352000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:34.354838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:34.368173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1155/2326] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:36.168302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:36.177093 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38162a4-9474-4b5f-9643-4e4802c1d15f service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:36.180910 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:36.181564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:36.181850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:36.182130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:36.183920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:36.194698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:36.196211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:36.206518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1172/2327] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:36.930755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:36.932208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:36.949055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51493274-dbf5-4289-a13a-acd6fe9f755c service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:36.966769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:36.968301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:36.968587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:36.968858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:36.971056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:36.985860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:36.987473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:37.000810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1156/2328] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:42.767378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:42.785144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b7e3e-3fac-49fd-ac48-4a7f549990eb service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:42.790887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:42.791535 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:42.791820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:42.792119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:42.793966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:42.807425 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:42.809072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:42.820890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1173/2329] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:46.995156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=37 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:55:47.123741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:55:47.135720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23582175-fe77-4819-b11b-4cad6ac396f6 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:55:47.141279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:55:47.141933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:55:47.142220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:55:47.142499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:55:47.144342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:55:47.157092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:55:47.158692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:55:47.170242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1157/2330] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:55:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:55:52.815452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=23 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:55:57.165545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:57:35.102931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:35.122491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf50409-f376-4343-9b52-22fbd450bb75 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:35.126388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:35.127022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:35.127306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:35.127582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:35.136543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:35.137977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:35.147074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1174/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:57:35.191508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:35.200507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203b5e8-0bcc-4929-8678-1ad836eee672 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:35.207760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:35.208450 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:35.208730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:35.209001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:35.218035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:35.219466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:35.248369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1158/2332] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:57:35.293032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:35.302240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36144c49-20f5-4147-81e7-303afb7204d8 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:35.305885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:35.306519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:35.306801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:35.307079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:35.315831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:35.317557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:35.345078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1175/2333] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:57:35.903593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:35.929536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee5167-66d0-4fbf-b21a-9f27f81523d3 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:35.934284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:35.935054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:35.935443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:35.935707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:35.937923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:57:36.011075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:36.012601 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:36.022902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1159/2334] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:57:39.001808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.014475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b7adb-7c37-4585-b037-26bd96af2a11 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.019142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.020464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.021005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.021356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.023198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:57:39.036359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.037988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.050546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1176/2335] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:57:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:57:39.057174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.059252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:57:39.074473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b7783-7fc1-41f8-bcc4-3e3c8761c7b3 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.079530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.080570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.081075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.081325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] PATH_INFO: `/v3/users/01120eedaa844f24a66024df8d3b2271` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.084329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01120eedaa844f24a66024df8d3b2271)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.086280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.138461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01120eedaa844f24a66024df8d3b2271'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.146731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01120eedaa844f24a66024df8d3b2271'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.147534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01120eedaa844f24a66024df8d3b2271'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.148260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01120eedaa844f24a66024df8d3b2271'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.156500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e71654d5-5788-4372-a00f-f26e9d8cc093 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01120eedaa844f24a66024df8d3b2271'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.174885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1160/2336] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 19:57:39 2022] DELETE /identity/v3/users/01120eedaa844f24a66024df8d3b2271 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:57:39.363225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.381524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3868c70-1a2f-42f1-b5d3-9871db722cbc service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.445337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.446209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.446504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.446775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] PATH_INFO: `/v3/projects/442be57bdd524fa092e7ebc12bd007a1` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.449289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] RBAC: Authorizing `identity:delete_project(project_id=442be57bdd524fa092e7ebc12bd007a1)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.451277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.508992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '442be57bdd524fa092e7ebc12bd007a1', 'request_id': 'req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.519175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '442be57bdd524fa092e7ebc12bd007a1', 'request_id': 'req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.527384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '442be57bdd524fa092e7ebc12bd007a1', 'request_id': 'req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.540818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] The token cache is being invalidate because project 442be57bdd524fa092e7ebc12bd007a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 19:57:39.541114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.541807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-893ba60a-0bfe-4345-b25c-c6e4bd4ef79a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 19:57:39.543379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1177/2337] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 19:57:39 2022] DELETE /identity/v3/projects/442be57bdd524fa092e7ebc12bd007a1 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:57:39.549173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.549468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.549749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.568538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:57:39.569736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:57:39.603455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:57:39.644065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1161/2338] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:57:39.653273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.653589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.653851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.672944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:57:39.674157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:57:39.682696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1178/2339] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:57:39.686973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.697135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2697f3-38e9-4ba9-a5d7-63c1c5441c41 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.701800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.702458 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.702739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.703013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.703747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 19:57:39.704052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.705704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.718514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1162/2340] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 19:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 19:57:39.723385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.732929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3158520-0f04-4f43-8ffa-dd8423db3697 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.737632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.739018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.739288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.739532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.740262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.742197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.762251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:57:39.771244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1179/2341] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 19:57:39 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 19:57:39.775661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.790360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74770b-cb22-4c2c-af21-34ee36d42d3a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.854310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.855023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.855310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.855622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.856340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.858289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.879343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1163/2342] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 19:57:39 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 19:57:39.883805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.895154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba6359-8196-4e92-b4d1-6725a8f1d364 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.900204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.900878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.901163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.901442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.902179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.903805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.916517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1180/2343] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 19:57:39.920842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.931713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd02f4-267f-406d-91ca-46036be40d21 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.936472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd6e92-3dd5-4ba2-9286-d7ca78bf4029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.937136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd6e92-3dd5-4ba2-9286-d7ca78bf4029 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.937421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd6e92-3dd5-4ba2-9286-d7ca78bf4029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.937699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd6e92-3dd5-4ba2-9286-d7ca78bf4029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.938435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd6e92-3dd5-4ba2-9286-d7ca78bf4029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.940036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd6e92-3dd5-4ba2-9286-d7ca78bf4029 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:39.951144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1164/2344] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 19:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 19:57:39.955713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:57:39.967536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965315e5-7e40-493c-9136-b2049f0c3402 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:57:39.972247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:57:39.974312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:39.974596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:39.974874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] PATH_INFO: `/v3/projects/e30cb4324b884c688a09a3b8ca015777/users/d115f08532bf46259e8fe732dbaa7586/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:39.990354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e30cb4324b884c688a09a3b8ca015777, user_id=d115f08532bf46259e8fe732dbaa7586, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:57:39.993182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:57:40.008436 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1181/2345] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 19:57:39 2022] PUT /identity/v3/projects/e30cb4324b884c688a09a3b8ca015777/users/d115f08532bf46259e8fe732dbaa7586/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 19:57:40.020931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:57:40.021240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:57:40.021515 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:57:40.052549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] MFA Rules not processed for user `d115f08532bf46259e8fe732dbaa7586`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 19:57:40.053837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:57:40.124058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1165/2346] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 19:57:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 19:57:50.004091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=16,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:57:50.120684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:58:03.115546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:58:03.117437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:58:03.202581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2470e6-23c3-4208-92d3-16d85de9aeae admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:58:03.206551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:58:03.207310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:58:03.207655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:58:03.207995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:58:03.218294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:58:03.219979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:58:03.231423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1182/2347] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:58:03 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:58:13.226277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=31 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:59:26.770556 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:59:26.771972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:59:26.843202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf922b-6689-414a-a76e-c34903fc7555 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:59:26.848011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:59:26.848687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:59:26.848972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:59:26.849253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:59:26.851172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:59:26.862837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:59:26.864483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:59:26.875308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1166/2348] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:59:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 19:59:36.111500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 19:59:36.112959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:59:36.124379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2182b-bbcd-462e-88e8-d5877cf86873 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 19:59:36.127933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 19:59:36.128606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 19:59:36.128894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 19:59:36.129174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 19:59:36.130944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 19:59:36.212182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 19:59:36.213649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 19:59:36.232809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1183/2349] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 19:59:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 19:59:36.870166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=32 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 19:59:46.229970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=41 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:01.703128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:01.714200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8547e-e7ca-4efc-80d6-9112f6602eb5 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:01.717783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:01.718548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:01.718930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:01.719100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:01.720916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:01.837960 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:01.841246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:01.865178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1167/2350] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:02.682809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:02.691860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96faa3fe-a067-49ba-b472-fd819d790f87 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:02.695446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:02.696106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:02.696397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:02.696676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:02.699069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:02.715305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:02.716944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:02.728431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1184/2351] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:00:03.363167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:03.372581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7a4c1-0313-472c-ba7b-65a61a0efdd5 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:03.376546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:03.377188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:03.377471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:03.377749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:03.379509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:03.469308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:03.470753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469cc045-09ab-4df5-89b0-e7b43e152dc5 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:03.490016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1168/2352] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:03.637237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:03.638920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:03.735219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb1e19-a030-44ac-9982-80356f1c14cf service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:03.739946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:03.740878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:03.741158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:03.741439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] PATH_INFO: `/v3/users/cc85b5d830dd4430a479c6977dbb45ad` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:03.742745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc85b5d830dd4430a479c6977dbb45ad)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:03.745307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:03.791994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc85b5d830dd4430a479c6977dbb45ad'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:03.799191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc85b5d830dd4430a479c6977dbb45ad'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:03.799803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc85b5d830dd4430a479c6977dbb45ad'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:03.800435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc85b5d830dd4430a479c6977dbb45ad'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:03.818117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc85b5d830dd4430a479c6977dbb45ad'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:03.831537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1185/2353] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:03 2022] DELETE /identity/v3/users/cc85b5d830dd4430a479c6977dbb45ad => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:03.836014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:03.836340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:03.836623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:03.864892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:03.866079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:03.878459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:03.926319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1169/2354] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:03.936656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:03.938079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:03.980620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:03.999832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aca2f8-26f6-431b-8344-42e5ab45a062 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:04.003469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:04.004356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:04.004641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:04.004931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:04.014626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:04.016463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:04.026392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1186/2355] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:04.303783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:04.305260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:04.317360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e192fd-c7aa-4e9b-856a-f64d9b21a89e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:04.328024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:04.328931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:04.329216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:04.329519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] PATH_INFO: `/v3/projects/0c0ef1b653a5430d8d1675991d907ac7` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:04.331017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c0ef1b653a5430d8d1675991d907ac7)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:04.333215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:04.384204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c0ef1b653a5430d8d1675991d907ac7', 'request_id': 'req-553e6085-a732-4f57-870b-3e047a1fc512'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.390648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c0ef1b653a5430d8d1675991d907ac7', 'request_id': 'req-553e6085-a732-4f57-870b-3e047a1fc512'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.396346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c0ef1b653a5430d8d1675991d907ac7', 'request_id': 'req-553e6085-a732-4f57-870b-3e047a1fc512'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.410467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] The token cache is being invalidate because project 0c0ef1b653a5430d8d1675991d907ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:04.410760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.411387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.412972 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1170/2356] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:04 2022] DELETE /identity/v3/projects/0c0ef1b653a5430d8d1675991d907ac7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:04.418283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:04.428957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd14a9c-62fd-4081-9135-382dc1726c06 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:04.484807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:04.485695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:04.485979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:04.486255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] PATH_INFO: `/v3/users/781f4e9ad9ea4825bc952eded5902235` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:04.487440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=781f4e9ad9ea4825bc952eded5902235)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:04.489429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:04.539592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '781f4e9ad9ea4825bc952eded5902235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.547894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '781f4e9ad9ea4825bc952eded5902235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.548536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '781f4e9ad9ea4825bc952eded5902235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.549148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '781f4e9ad9ea4825bc952eded5902235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.561755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5f00a177-3c94-4ece-bbd7-19761d868581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '781f4e9ad9ea4825bc952eded5902235'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.574334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1187/2357] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:04 2022] DELETE /identity/v3/users/781f4e9ad9ea4825bc952eded5902235 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:04.849677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:04.851157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:04.866420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e6085-a732-4f57-870b-3e047a1fc512 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:04.925958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:04.926997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:04.927176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:04.927406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] PATH_INFO: `/v3/projects/6d6ea528e2184e9199cec4b6eadb9722` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:04.928633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6ea528e2184e9199cec4b6eadb9722)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:04.930568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:04.975817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:04.979860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6ea528e2184e9199cec4b6eadb9722', 'request_id': 'req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.987246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6ea528e2184e9199cec4b6eadb9722', 'request_id': 'req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:04.993075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6ea528e2184e9199cec4b6eadb9722', 'request_id': 'req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:05.005418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] The token cache is being invalidate because project 6d6ea528e2184e9199cec4b6eadb9722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:05.005704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:05.006348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4c6d3cee-a109-434c-9c40-eb0418ea5fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:05.007880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1171/2358] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:04 2022] DELETE /identity/v3/projects/6d6ea528e2184e9199cec4b6eadb9722 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:05.013536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.014008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.014126 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.060198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:05.061386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:05.095070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:05.127564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1188/2359] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:05.135511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.135802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.136098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.155060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:05.156289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:05.164165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1172/2360] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:00:05.169312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.179237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf389ca-f553-40ca-968e-5ecf836384b1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.184429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.185123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.185407 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.185763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.186497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:05.186944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.188617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.200905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1189/2361] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:05.205124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.214799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd43087-83ba-4bd1-b6c2-61a7f77ffb12 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.219401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.220040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.220353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.220627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.221316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.223204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.261236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1173/2362] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/projects => generated 355 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:05.262331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.272331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d305605-1a41-41bd-bac9-a50b610028fc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.332289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.332917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.333285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.333487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.334169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.336123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.356727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1190/2363] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/users => generated 278 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:05.360926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.371034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ec417e-aa24-4677-ac83-86e6dfd09998 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.375977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.376661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.376943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.377215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.377936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.379530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.390225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1174/2364] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:05.394127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.404907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12660552-f252-4f65-b7c9-5a92c408c21d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.409488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6861fa-6eb6-4459-a583-7f60308c5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.410142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6861fa-6eb6-4459-a583-7f60308c5647 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.410429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6861fa-6eb6-4459-a583-7f60308c5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.410710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6861fa-6eb6-4459-a583-7f60308c5647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.411437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6861fa-6eb6-4459-a583-7f60308c5647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.413082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6861fa-6eb6-4459-a583-7f60308c5647 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.422840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1191/2365] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:05.426880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.437354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c03f0a-03ac-47b4-af72-85b8b557ca2d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.441982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.442864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.443149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.443427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] PATH_INFO: `/v3/projects/59b33e9e85d74f1d9ce535ae1e567677/users/04fb3d4615274854adbe83f628ebce63/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.458488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59b33e9e85d74f1d9ce535ae1e567677, user_id=04fb3d4615274854adbe83f628ebce63, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.461141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.474610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1175/2366] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:05 2022] PUT /identity/v3/projects/59b33e9e85d74f1d9ce535ae1e567677/users/04fb3d4615274854adbe83f628ebce63/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:05.485725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.486049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.486324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.514680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] MFA Rules not processed for user `04fb3d4615274854adbe83f628ebce63`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:05.515944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:05.584155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1192/2367] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:05.589326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.651232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b832c-0fb0-49be-8a4e-6d5fd7310985 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.656822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.657482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.657766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.658045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.658748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.660714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.685821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1176/2368] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/projects => generated 355 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:05.689767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.701215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064da21-4bce-48b1-af4b-38c06fec521f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.742987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:05.776348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.776661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.776945 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.777224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.777915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.779850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.799285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1193/2369] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:05 2022] POST /identity/v3/users => generated 284 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:05.803301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.813796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ccad65-9d4a-4565-a4fc-d450a4488e6b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.818409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.819102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.819386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.819662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.820417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.822019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.834387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1177/2370] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:05.839130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.849024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c9008b-5a6d-4d86-9366-c063c1e91fb4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.854248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.855124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.855406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.855679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] PATH_INFO: `/v3/projects/f300ebfc8f3e43148fda061da05a6a6f/users/50c799f7bffb486bac7fddcd8ba0e02f/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.870473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f300ebfc8f3e43148fda061da05a6a6f, user_id=50c799f7bffb486bac7fddcd8ba0e02f, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.873468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.886015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1194/2371] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:05 2022] PUT /identity/v3/projects/f300ebfc8f3e43148fda061da05a6a6f/users/50c799f7bffb486bac7fddcd8ba0e02f/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:05.890049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.900912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4ca3f-8450-404f-87f2-c06678d4ee7b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.930427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:05.960323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e4c30-4c48-440e-84b6-84647cad8fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.961000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e4c30-4c48-440e-84b6-84647cad8fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.961286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e4c30-4c48-440e-84b6-84647cad8fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.961595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e4c30-4c48-440e-84b6-84647cad8fe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.962329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e4c30-4c48-440e-84b6-84647cad8fe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:05.965046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e4c30-4c48-440e-84b6-84647cad8fe1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:05.976006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1178/2372] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:05 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:05.980878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:05.991107 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1912772c-a3cd-43da-9f07-805a18c85e39 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:05.996106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:05.997001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:05.997304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:05.997584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] PATH_INFO: `/v3/projects/f300ebfc8f3e43148fda061da05a6a6f/users/50c799f7bffb486bac7fddcd8ba0e02f/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:05.999794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f300ebfc8f3e43148fda061da05a6a6f, user_id=50c799f7bffb486bac7fddcd8ba0e02f, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:06.002496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:06.013416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1195/2373] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:05 2022] PUT /identity/v3/projects/f300ebfc8f3e43148fda061da05a6a6f/users/50c799f7bffb486bac7fddcd8ba0e02f/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:06.024817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:06.025111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:06.025953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:06.054772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] MFA Rules not processed for user `50c799f7bffb486bac7fddcd8ba0e02f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:06.056140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:06.127669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1179/2374] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:06 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:06.139875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:06.141625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:06.200619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d201ef1-6be2-40c0-aa4e-e77e0eea9e95 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:06.204269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:06.204918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:06.205202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:06.205712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:06.215821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:06.217306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:06.227175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1196/2375] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:06 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:06.299554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:06.309441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c77b-beb4-4735-84b0-d20938b07956 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:06.313047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:06.313689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:06.313973 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:06.314250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:06.371427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:06.372903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:06.381963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1180/2376] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:06 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:08.457808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:08.467546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428fb5a-bf02-4788-a3fb-57906acdab19 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:08.471339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:08.472009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:08.472324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:08.472601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:08.474403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:08.534720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:08.536260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c2fc09-e07b-4820-a735-4b208abb442d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:08.545537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1197/2377] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:14.219729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:14.221286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:14.276808 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:14.290317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4bd487-28d6-43a8-922d-514e93d5ca3b service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:14.296330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:14.297149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:14.297435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:14.297710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] PATH_INFO: `/v3/users/d115f08532bf46259e8fe732dbaa7586` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:14.298925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d115f08532bf46259e8fe732dbaa7586)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:14.300898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:14.350535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd115f08532bf46259e8fe732dbaa7586'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.359238 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd115f08532bf46259e8fe732dbaa7586'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.359862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd115f08532bf46259e8fe732dbaa7586'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.360509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd115f08532bf46259e8fe732dbaa7586'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.367615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd115f08532bf46259e8fe732dbaa7586'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.385923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1181/2378] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:14 2022] DELETE /identity/v3/users/d115f08532bf46259e8fe732dbaa7586 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:14.390192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:14.390649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:14.390788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:14.409059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:14.410258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:14.432348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:14.476291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1198/2379] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:14.488361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:14.489804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:14.552033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9da734-2c82-4b47-85d4-364358e61a9f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:14.555753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156763cc-89c0-4453-8587-1827a08e0835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:14.556322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156763cc-89c0-4453-8587-1827a08e0835 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:14.556607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156763cc-89c0-4453-8587-1827a08e0835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:14.556985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156763cc-89c0-4453-8587-1827a08e0835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:14.566567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156763cc-89c0-4453-8587-1827a08e0835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:14.568008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156763cc-89c0-4453-8587-1827a08e0835 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:14.578594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1182/2380] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:14.874816 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:14.876352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:14.892696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a89db7f-46e6-401c-8279-59bbd1e1251c None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:14.897439 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:14.898555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:14.898906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:14.899245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] PATH_INFO: `/v3/projects/e30cb4324b884c688a09a3b8ca015777` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:14.902278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e30cb4324b884c688a09a3b8ca015777)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:14.904877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:14.959521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e30cb4324b884c688a09a3b8ca015777', 'request_id': 'req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.967595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e30cb4324b884c688a09a3b8ca015777', 'request_id': 'req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.973833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e30cb4324b884c688a09a3b8ca015777', 'request_id': 'req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.984994 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] The token cache is being invalidate because project e30cb4324b884c688a09a3b8ca015777 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:14.985277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.985891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3547a3ae-07b1-4ea0-81f2-84d33d323ccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:14.987402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1199/2381] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:14 2022] DELETE /identity/v3/projects/e30cb4324b884c688a09a3b8ca015777 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:14.992246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:14.992541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:14.992824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.011206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:15.012428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:15.071576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1183/2382] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:15.079778 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.079908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.080293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.098721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:15.099982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:15.108764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1200/2383] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:15.112937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.123382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905db83f-f1de-4b42-b913-b6031db1f1fa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.128019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.128709 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.128999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.129285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.130024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:15.130341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.131945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.144729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1184/2384] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:15.149036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.158909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb874a96-0424-4019-9378-b5fcf4b96df5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.163618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.164308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.164595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.164869 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.165598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.168283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.197344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1201/2385] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:15 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:15.198769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.204105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:15.210267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1383a12d-89c7-4b12-8be4-333aff7f7037 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.270211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.270912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.271147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.271424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.272134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.274063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.294470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1185/2386] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:15 2022] POST /identity/v3/users => generated 302 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:15.298607 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.309585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cee96b-8efc-48c8-9773-43b521f132cf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.314224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.314874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.315153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.315429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.316189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.317782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.328212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1202/2387] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:15.332655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.343151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913fddf-bcb4-4b61-8a13-e2f027cdfe2f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.347967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a721d-c0e0-436e-a0ec-122c264875f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.348653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a721d-c0e0-436e-a0ec-122c264875f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.348944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a721d-c0e0-436e-a0ec-122c264875f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.349225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a721d-c0e0-436e-a0ec-122c264875f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.350017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a721d-c0e0-436e-a0ec-122c264875f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.351713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a721d-c0e0-436e-a0ec-122c264875f3 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.362560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1186/2388] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:15.366546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.377826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8224c78e-cfff-4a33-aa99-8e9e1632ab8a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.382495 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.383396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.383677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.383955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] PATH_INFO: `/v3/projects/2342da3572df456f878c3efc13c467bd/users/b6d3d9d34df04d298acb0de125ac3aff/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.399566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2342da3572df456f878c3efc13c467bd, user_id=b6d3d9d34df04d298acb0de125ac3aff, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.402488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.418039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1203/2389] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:15 2022] PUT /identity/v3/projects/2342da3572df456f878c3efc13c467bd/users/b6d3d9d34df04d298acb0de125ac3aff/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:15.429628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.430007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.430336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.457203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] MFA Rules not processed for user `b6d3d9d34df04d298acb0de125ac3aff`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:15.458416 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:15.526014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1187/2390] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:15.530619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.576316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:15.593838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7b969-78e7-4ea8-9591-36bda7db95cb admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.598417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.599066 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.599345 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.599618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.600337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.602243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.629794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1204/2391] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:15 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:15.633841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.645162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab20032-41ce-489d-a3be-5016668e1ec5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.701938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.702603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.702887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.703161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.703846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.705811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.719987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:15.728425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1188/2392] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:15 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:15.732360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.742646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81dea3-6740-407b-82ad-4b935632f5b6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.748681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.749437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.749714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.749990 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.750729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.752599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.764843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1205/2393] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:15.769023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.779609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8456cbfe-7313-4396-a566-566fec48847e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.784411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.785301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.785594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.785862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] PATH_INFO: `/v3/projects/012fdceb89034cb5b52dd0a3227dada1/users/b23c307182d44dab91265c411a9c44b5/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.801463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012fdceb89034cb5b52dd0a3227dada1, user_id=b23c307182d44dab91265c411a9c44b5, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.804160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.818719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1189/2394] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:15 2022] PUT /identity/v3/projects/012fdceb89034cb5b52dd0a3227dada1/users/b23c307182d44dab91265c411a9c44b5/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:15.823580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.834303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee47f7a-c052-42c4-a39d-2ba1b5a25e3d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.895674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526f762-28b1-4cf0-bcbe-7ac397df9e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.896518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526f762-28b1-4cf0-bcbe-7ac397df9e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.896802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526f762-28b1-4cf0-bcbe-7ac397df9e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.897173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526f762-28b1-4cf0-bcbe-7ac397df9e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.897998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526f762-28b1-4cf0-bcbe-7ac397df9e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.899814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526f762-28b1-4cf0-bcbe-7ac397df9e2c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.910762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1206/2395] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:00:15.914901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:15.927024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361117f-6b68-43d0-b281-af67fc79c1d1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:15.931686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:15.932873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:15.933257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:15.933544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] PATH_INFO: `/v3/projects/012fdceb89034cb5b52dd0a3227dada1/users/b23c307182d44dab91265c411a9c44b5/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:15.935577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=012fdceb89034cb5b52dd0a3227dada1, user_id=b23c307182d44dab91265c411a9c44b5, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:15.938299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:15.950261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1190/2396] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:15 2022] PUT /identity/v3/projects/012fdceb89034cb5b52dd0a3227dada1/users/b23c307182d44dab91265c411a9c44b5/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:16.013260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:16.013573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:16.013840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:16.051378 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] MFA Rules not processed for user `b23c307182d44dab91265c411a9c44b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:16.052653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:16.153404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1207/2397] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:16.168440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:16.170474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:16.261877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773eb0d1-46c4-4238-a03c-1411921653f0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:16.267881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489db93c-e606-4141-876b-de028a68d4f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:16.269036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489db93c-e606-4141-876b-de028a68d4f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:16.269531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489db93c-e606-4141-876b-de028a68d4f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:16.269930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489db93c-e606-4141-876b-de028a68d4f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:16.282965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489db93c-e606-4141-876b-de028a68d4f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:16.284672 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489db93c-e606-4141-876b-de028a68d4f9 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:16.297821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1191/2398] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:16.476952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:16.527859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:16.554450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be8d1c-9887-4fad-b9dd-6bd1f9982c21 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:16.560754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:16.561677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:16.561958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:16.562237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] PATH_INFO: `/v3/users/b6d3d9d34df04d298acb0de125ac3aff` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:16.563463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d3d9d34df04d298acb0de125ac3aff)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:16.565731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:16.624234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d3d9d34df04d298acb0de125ac3aff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:16.635755 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d3d9d34df04d298acb0de125ac3aff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:16.637099 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d3d9d34df04d298acb0de125ac3aff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:16.637758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d3d9d34df04d298acb0de125ac3aff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:16.653479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d3d9d34df04d298acb0de125ac3aff'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:16.670197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1208/2399] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:16 2022] DELETE /identity/v3/users/b6d3d9d34df04d298acb0de125ac3aff => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:16.675223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:16.675518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:16.675800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:16.699607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:16.700858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:16.779729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1192/2400] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:16.790105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:16.791641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:16.862003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25971c-9f85-4ef3-8243-ccc1aead12aa admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:16.866199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:16.866856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:16.867140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:16.867411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:16.879039 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:16.880544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:16.894679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1209/2401] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:17.216470 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:17.218117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:17.237510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2be57-4733-4c14-bffc-5183faa6bdbd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:17.242993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:17.244050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:17.244848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:17.245336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] PATH_INFO: `/v3/projects/2342da3572df456f878c3efc13c467bd` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:17.247085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2342da3572df456f878c3efc13c467bd)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:17.249429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:17.305904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2342da3572df456f878c3efc13c467bd', 'request_id': 'req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.314179 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2342da3572df456f878c3efc13c467bd', 'request_id': 'req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.319472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:17.321117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2342da3572df456f878c3efc13c467bd', 'request_id': 'req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.339385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] The token cache is being invalidate because project 2342da3572df456f878c3efc13c467bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:17.339989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.340966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.342984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1193/2402] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:17 2022] DELETE /identity/v3/projects/2342da3572df456f878c3efc13c467bd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:17.347390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:17.358040 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c771e2ee-e323-4927-bb0a-eab2853d4894 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:17.424608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:17.425636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:17.426092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:17.426255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] PATH_INFO: `/v3/users/b23c307182d44dab91265c411a9c44b5` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:17.427739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23c307182d44dab91265c411a9c44b5)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:17.430454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:17.433552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:17.488106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23c307182d44dab91265c411a9c44b5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.497333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23c307182d44dab91265c411a9c44b5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.498265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23c307182d44dab91265c411a9c44b5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.498944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23c307182d44dab91265c411a9c44b5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.514487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-883036c5-5bfa-46c5-ac39-352c1e44cc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23c307182d44dab91265c411a9c44b5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:17.536695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1210/2403] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:17 2022] DELETE /identity/v3/users/b23c307182d44dab91265c411a9c44b5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:17.905395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:17.906877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:17.921434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaec95-59ca-40ba-a4d8-31f28d6eba9d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:17.985234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:17.986593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:17.986891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:17.987169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] PATH_INFO: `/v3/projects/012fdceb89034cb5b52dd0a3227dada1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:17.988380 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012fdceb89034cb5b52dd0a3227dada1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:17.990300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.038780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012fdceb89034cb5b52dd0a3227dada1', 'request_id': 'req-ef8ff900-c9fe-4783-b6f4-126c13c32954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:18.045572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012fdceb89034cb5b52dd0a3227dada1', 'request_id': 'req-ef8ff900-c9fe-4783-b6f4-126c13c32954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:18.051121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012fdceb89034cb5b52dd0a3227dada1', 'request_id': 'req-ef8ff900-c9fe-4783-b6f4-126c13c32954'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:18.063479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] The token cache is being invalidate because project 012fdceb89034cb5b52dd0a3227dada1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:18.063771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:18.064472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ef8ff900-c9fe-4783-b6f4-126c13c32954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:18.066294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1194/2404] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:17 2022] DELETE /identity/v3/projects/012fdceb89034cb5b52dd0a3227dada1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:18.071186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.071478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.071757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.089957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:18.092417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:18.153428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1211/2405] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:18.161367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.161659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.161940 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.180773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:18.182594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:18.190487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1195/2406] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:18.200350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.210170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c6dd95-8708-4bf6-9f24-fadbe1574c5f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.214877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.215549 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.215830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.216129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.216866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:18.217169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.218803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.231871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1212/2407] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:18.236582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.246742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e519-2ecc-4061-b28f-6ddce7582344 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.252109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.252784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.253100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.253376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.254086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.256010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.283863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1196/2408] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:18.287292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.297805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a50c6-da5c-409e-ae55-386ab8e377e5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.361413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.362082 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.362367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.362650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.363403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.365429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.381538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:18.390374 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1213/2409] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:18.394573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.404924 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4503d3-7454-4a97-95d0-5e2d9da4be91 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.409659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.410312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.410597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.410881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.411603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.413251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.424050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1197/2410] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:18.427943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.438581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbb03c-51d5-4e17-8f35-31d362e46326 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.443183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1394a0-eafd-4ce1-a52b-e13ae771a70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.443840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1394a0-eafd-4ce1-a52b-e13ae771a70c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.444154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1394a0-eafd-4ce1-a52b-e13ae771a70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.444435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1394a0-eafd-4ce1-a52b-e13ae771a70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.445157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1394a0-eafd-4ce1-a52b-e13ae771a70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.446909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1394a0-eafd-4ce1-a52b-e13ae771a70c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.457087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1214/2411] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:18.461168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.471968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95de982-2377-4c45-b38b-d5321eb7b80a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.477029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.477911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.478192 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.479071 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] PATH_INFO: `/v3/projects/2b6cf5094be9478ebd0c690849b7cd51/users/a1fe476c05784cf495602c9a06b355cf/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.495540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b6cf5094be9478ebd0c690849b7cd51, user_id=a1fe476c05784cf495602c9a06b355cf, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.498185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.512617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1198/2412] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:18 2022] PUT /identity/v3/projects/2b6cf5094be9478ebd0c690849b7cd51/users/a1fe476c05784cf495602c9a06b355cf/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:18.524266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.524791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.524923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.555272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] MFA Rules not processed for user `a1fe476c05784cf495602c9a06b355cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:18.557119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:18.632737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1215/2413] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:18.638143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.660491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:18.708615 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf133b4-fe9c-4637-8262-103168161040 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.713381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.713991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.714353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.714533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.715232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.717213 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.743086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1199/2414] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:18.747813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.758584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579eae3-20d7-454e-9438-629e0519d8c8 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.826588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.827311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.827597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.827875 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.828610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.830548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.855863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1216/2415] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:18 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:18.860394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.872525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a2091-2c3a-4eb6-b19a-1d4f0e625fed admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.878009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.878817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.879107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.879383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.880147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.881746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.897845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1200/2416] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:00:18.902959 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.913984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37177f3b-6837-4dd5-898b-582d9b514c2c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:18.919002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:18.919925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:18.920252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:18.920555 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] PATH_INFO: `/v3/projects/54c73c4de5ac407586d174e2df804562/users/9e4199bc39054df0bffcddcc2054c92c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:18.935269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c73c4de5ac407586d174e2df804562, user_id=9e4199bc39054df0bffcddcc2054c92c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:18.938034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:18.953285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1217/2417] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:18 2022] PUT /identity/v3/projects/54c73c4de5ac407586d174e2df804562/users/9e4199bc39054df0bffcddcc2054c92c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:18.957422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:18.970814 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c3bfd-c791-4db8-af04-56d28fea5ebc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:19.028119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cba151-5249-4384-91b4-afbd3c3f7f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:19.028718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba151-5249-4384-91b4-afbd3c3f7f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:19.029000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba151-5249-4384-91b4-afbd3c3f7f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:19.029277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cba151-5249-4384-91b4-afbd3c3f7f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:19.029992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba151-5249-4384-91b4-afbd3c3f7f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:19.031596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cba151-5249-4384-91b4-afbd3c3f7f95 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:19.041890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1201/2418] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:00:19.046116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:19.056290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033747f5-f4d6-4bf0-9518-e2968698a294 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:19.061694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:19.062574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:19.062857 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:19.063134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] PATH_INFO: `/v3/projects/54c73c4de5ac407586d174e2df804562/users/9e4199bc39054df0bffcddcc2054c92c/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:19.065139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c73c4de5ac407586d174e2df804562, user_id=9e4199bc39054df0bffcddcc2054c92c, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:19.067788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:19.077929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1218/2419] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:19 2022] PUT /identity/v3/projects/54c73c4de5ac407586d174e2df804562/users/9e4199bc39054df0bffcddcc2054c92c/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:19.088165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:19.088506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:19.088840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:19.116305 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] MFA Rules not processed for user `9e4199bc39054df0bffcddcc2054c92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:19.117577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:19.189106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1202/2420] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:19.200653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:19.202357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:19.236390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:19.266124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9afc42-59b2-4988-aa23-83e4f67ee78e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:19.270125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:19.270868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:19.271147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:19.271417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:19.283489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:19.284961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:19.295547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1219/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:19.366579 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:19.376602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c082130-5574-4fe6-83a1-b9f883bcb88e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:19.380181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:19.380819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:19.381103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:19.381382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:19.389704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:19.439537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:19.441014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:19.449967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1203/2422] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:21.089619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:21.090996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:21.149516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaace07a-0604-4a22-a762-0745272f0b1e service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:21.153363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:21.154005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:21.154287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:21.154565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:21.156373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:21.225134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:21.226518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:21.237531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1220/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:22.014833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.016447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:22.035919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662be98-887a-480e-aa60-8403320ebabe service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.040479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.041381 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.041844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.042045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] PATH_INFO: `/v3/users/918edf180b5b40f58994469fb37d25c2` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.056718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] RBAC: Authorizing `identity:delete_user(user_id=918edf180b5b40f58994469fb37d25c2)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.058944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.110686 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918edf180b5b40f58994469fb37d25c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.118094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918edf180b5b40f58994469fb37d25c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.118791 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918edf180b5b40f58994469fb37d25c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.119597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918edf180b5b40f58994469fb37d25c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.127666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9af563a6-307b-4205-b913-c6e08cf5d34a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918edf180b5b40f58994469fb37d25c2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.153016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1204/2424] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:22 2022] DELETE /identity/v3/users/918edf180b5b40f58994469fb37d25c2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:22.348521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.361153 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d13b9-164a-46a9-ba25-d992ecdb6df7 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.426385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.427279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.427565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.427848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] PATH_INFO: `/v3/projects/7998a53aaca0417aa89d905f03b923c6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.429088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7998a53aaca0417aa89d905f03b923c6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.431022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.458958 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:22.485073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7998a53aaca0417aa89d905f03b923c6', 'request_id': 'req-1a4f460b-7646-4caf-9aea-f08088684b06'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.493542 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7998a53aaca0417aa89d905f03b923c6', 'request_id': 'req-1a4f460b-7646-4caf-9aea-f08088684b06'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.500748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7998a53aaca0417aa89d905f03b923c6', 'request_id': 'req-1a4f460b-7646-4caf-9aea-f08088684b06'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.513008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] The token cache is being invalidate because project 7998a53aaca0417aa89d905f03b923c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:22.513417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.514219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1a4f460b-7646-4caf-9aea-f08088684b06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:22.516009 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1221/2425] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:22 2022] DELETE /identity/v3/projects/7998a53aaca0417aa89d905f03b923c6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:22.521412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.521805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.522189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.541941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:22.543160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:22.607287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1205/2426] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:00:22.614359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.614753 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.615041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.633847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:22.635746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:22.644758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1222/2427] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:22.649064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.659964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7efaf-8093-4896-867f-d419aa7cabc3 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.664563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.665249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.665528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.665804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.666627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:22.666931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.668551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.676133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:22.682650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1206/2428] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:22.687590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.697010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b809e3-e68f-4af2-96d0-72eefc88496b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.702732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.703386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.703670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.703947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.704670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.706626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.731873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1223/2429] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:22 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:22.736438 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.747214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9bc68-581b-438b-a470-c191d189cb1e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.813705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.814385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.814669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.815121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.815820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.817970 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.842548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1207/2430] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:22 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:22.843244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.855300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79905d33-d533-4f78-9dbf-2f182f4990cc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.861086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.861820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.862109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.862385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.863189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.864843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.876540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1224/2431] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:22.880624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.893281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00551a38-00a6-4481-afc6-3d1b83378141 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.908593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310cb12-35dc-456f-be5f-21af18f4baeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.908593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310cb12-35dc-456f-be5f-21af18f4baeb admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.908593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310cb12-35dc-456f-be5f-21af18f4baeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.908593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310cb12-35dc-456f-be5f-21af18f4baeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.909160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310cb12-35dc-456f-be5f-21af18f4baeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.909160 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310cb12-35dc-456f-be5f-21af18f4baeb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.914610 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1208/2432] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:22.918927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:22.929732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8088f0-9cc6-470c-ab37-6adfe5c19980 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:22.935594 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:22.936524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.936785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.937063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] PATH_INFO: `/v3/projects/3049f8ee616f47088a44139a8ce036c1/users/bf62362e4f8c4901b4c9f5001f12a126/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:22.955348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3049f8ee616f47088a44139a8ce036c1, user_id=bf62362e4f8c4901b4c9f5001f12a126, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:22.957942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:22.972203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1225/2433] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:22 2022] PUT /identity/v3/projects/3049f8ee616f47088a44139a8ce036c1/users/bf62362e4f8c4901b4c9f5001f12a126/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:22.983379 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:22.983670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:22.983955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.013964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] MFA Rules not processed for user `bf62362e4f8c4901b4c9f5001f12a126`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:23.015189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:23.038457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:23.040238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:23.085702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1209/2434] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:23.096276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:23.097620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:23.105325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b3e18e-fcf4-4749-9153-0ce3bb24c475 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:23.110184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:23.111063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:23.111359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:23.111626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] PATH_INFO: `/v3/users/04fb3d4615274854adbe83f628ebce63` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.112900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=04fb3d4615274854adbe83f628ebce63)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:23.114842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:23.156649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee6236-2be2-4c27-b375-d2c04fb71822 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:23.160481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490b3437-4ae9-49cc-95ca-5dc027a70f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:23.161132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b3437-4ae9-49cc-95ca-5dc027a70f44 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:23.161492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b3437-4ae9-49cc-95ca-5dc027a70f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:23.161701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490b3437-4ae9-49cc-95ca-5dc027a70f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.164129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04fb3d4615274854adbe83f628ebce63'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.166849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:23.170655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b3437-4ae9-49cc-95ca-5dc027a70f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:23.172224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490b3437-4ae9-49cc-95ca-5dc027a70f44 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:23.172903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04fb3d4615274854adbe83f628ebce63'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.173578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04fb3d4615274854adbe83f628ebce63'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.174579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04fb3d4615274854adbe83f628ebce63'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.182785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1210/2435] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:00:23.188837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04fb3d4615274854adbe83f628ebce63'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.202076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1226/2436] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:23 2022] DELETE /identity/v3/users/04fb3d4615274854adbe83f628ebce63 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:23.206541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:23.206836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:23.207118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.226758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:23.228048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:23.233419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:23.297987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1211/2437] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:23.307905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:23.309363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:23.377543 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f5096-19e6-410d-b0b6-2e85f4d4b9ce admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:23.381654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:23.382300 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:23.382718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:23.382826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.393460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:23.395416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:23.406455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1227/2438] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:00:23.703296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:23.704807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:23.729178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1d46f-cc0e-4ab2-b792-fede91d94197 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:23.734371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:23.735241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:23.735526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:23.735803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] PATH_INFO: `/v3/projects/59b33e9e85d74f1d9ce535ae1e567677` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.737038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59b33e9e85d74f1d9ce535ae1e567677)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:23.739012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:23.791219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59b33e9e85d74f1d9ce535ae1e567677', 'request_id': 'req-de523f17-01a4-45f0-b5be-93d8335e7518'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.797304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59b33e9e85d74f1d9ce535ae1e567677', 'request_id': 'req-de523f17-01a4-45f0-b5be-93d8335e7518'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.802736 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59b33e9e85d74f1d9ce535ae1e567677', 'request_id': 'req-de523f17-01a4-45f0-b5be-93d8335e7518'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.816215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] The token cache is being invalidate because project 59b33e9e85d74f1d9ce535ae1e567677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:23.816585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.821699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.824250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1212/2439] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:23 2022] DELETE /identity/v3/projects/59b33e9e85d74f1d9ce535ae1e567677 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:23.828242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:23.838465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51dfd0-4ecc-408b-a3bf-ef8e532d6ad9 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:23.911440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:23.912468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:23.912824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:23.913405 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] PATH_INFO: `/v3/users/50c799f7bffb486bac7fddcd8ba0e02f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:23.914765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50c799f7bffb486bac7fddcd8ba0e02f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:23.917022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:23.975101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50c799f7bffb486bac7fddcd8ba0e02f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.983308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50c799f7bffb486bac7fddcd8ba0e02f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.984049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50c799f7bffb486bac7fddcd8ba0e02f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.984824 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50c799f7bffb486bac7fddcd8ba0e02f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:23.998758 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5358da85-56bc-49f8-9919-a77e8afa5234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50c799f7bffb486bac7fddcd8ba0e02f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:24.000002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:24.014545 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1228/2440] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:23 2022] DELETE /identity/v3/users/50c799f7bffb486bac7fddcd8ba0e02f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:24.321402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.322912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:24.344688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de523f17-01a4-45f0-b5be-93d8335e7518 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.403868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.404766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.405066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.405343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] PATH_INFO: `/v3/projects/f300ebfc8f3e43148fda061da05a6a6f` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.406524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=f300ebfc8f3e43148fda061da05a6a6f)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.408501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.446568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:24.459127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f300ebfc8f3e43148fda061da05a6a6f', 'request_id': 'req-b4102df9-2f57-4a3e-bf99-784718677cfa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:24.466342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f300ebfc8f3e43148fda061da05a6a6f', 'request_id': 'req-b4102df9-2f57-4a3e-bf99-784718677cfa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:24.473081 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f300ebfc8f3e43148fda061da05a6a6f', 'request_id': 'req-b4102df9-2f57-4a3e-bf99-784718677cfa'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:24.486851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] The token cache is being invalidate because project f300ebfc8f3e43148fda061da05a6a6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:24.487203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:24.487938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b4102df9-2f57-4a3e-bf99-784718677cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:24.489683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1213/2441] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:24 2022] DELETE /identity/v3/projects/f300ebfc8f3e43148fda061da05a6a6f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:24.494810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.496376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.496732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.515756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:24.517011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:24.577295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1229/2442] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:24.585090 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.585386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.585673 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.603520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:24.604777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:24.612565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1214/2443] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:24.616670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.627690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1db2e5-2190-488b-9905-000c44efcb98 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.632184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.632832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.633114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.633388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.634108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:24.634413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.637073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.648739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1230/2444] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:24.653032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.662957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f21475-dd8a-4953-bab8-32af11636f68 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.668032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.668902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.669187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.669490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.670191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.672127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.699943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1215/2445] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:24 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:24.704075 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.715045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976012bd-1007-44c5-8c33-660070bae07d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.773536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.774493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.774493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.774748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.775449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.777435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.802013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1231/2446] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:24 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:24.803146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.814441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558bd484-79db-49d1-bde6-4e8b4e5cffef admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.819654 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.820343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.820627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.820907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.821712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.823894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.835293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1216/2447] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:00:24.839391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.849529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10f5855-db1d-41e9-a369-4855229a6189 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.854213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103edc92-b9ab-4778-a1bb-f7123164e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.855609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103edc92-b9ab-4778-a1bb-f7123164e440 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.855896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103edc92-b9ab-4778-a1bb-f7123164e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.856219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103edc92-b9ab-4778-a1bb-f7123164e440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.856952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103edc92-b9ab-4778-a1bb-f7123164e440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.858569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103edc92-b9ab-4778-a1bb-f7123164e440 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.868650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1232/2448] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:24.873800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:24.883701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e955954-975f-4689-b681-5a22f314cbcf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:24.888543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:24.889418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.889701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.889975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] PATH_INFO: `/v3/projects/3b6933e4365243d1b4a68abada8f6d83/users/382e8ab4d0f646cd93a7f9025a454e6a/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.904248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b6933e4365243d1b4a68abada8f6d83, user_id=382e8ab4d0f646cd93a7f9025a454e6a, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:24.906907 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:24.920325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1217/2449] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:24 2022] PUT /identity/v3/projects/3b6933e4365243d1b4a68abada8f6d83/users/382e8ab4d0f646cd93a7f9025a454e6a/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:24.931453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:24.931749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:24.932073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:24.959170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] MFA Rules not processed for user `382e8ab4d0f646cd93a7f9025a454e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:24.960379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:25.006047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:25.033200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1233/2450] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:24 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:25.038275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.104038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60c44a5-df4d-44ff-ac38-1e82f42e2547 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.110067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.110839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.111215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.111557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.112408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.114658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.142346 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1218/2451] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:25 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:25.146462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.157794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c669b75-c928-4a94-9fcd-6a6653736ec2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.213304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.213911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.214195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.214548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.215226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.217273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.236315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1234/2452] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:25 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:25.240248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.251525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ead09-2f18-42ff-905f-9f2b6c03f842 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.256376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.257033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.257321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.257599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.258324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.259955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.271089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1219/2453] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:25.275147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.285367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4212a1-b77d-4d92-9185-1fe6353a12df admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.289970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.290853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.291143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.291424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] PATH_INFO: `/v3/projects/84f609a1cac14d03aad1b4f5a8e8306b/users/f069f64418dd43b2b7a644db1074b5d6/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.308517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f609a1cac14d03aad1b4f5a8e8306b, user_id=f069f64418dd43b2b7a644db1074b5d6, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.311410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.325726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1235/2454] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:25 2022] PUT /identity/v3/projects/84f609a1cac14d03aad1b4f5a8e8306b/users/f069f64418dd43b2b7a644db1074b5d6/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:25.329637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.340340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6951b2-3965-4b3b-9cf0-f91187058d5a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.370494 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:25.402467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bc4db-eefd-444f-bb59-f5ebbbf4f32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.403136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc4db-eefd-444f-bb59-f5ebbbf4f32c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.403420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc4db-eefd-444f-bb59-f5ebbbf4f32c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.403703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bc4db-eefd-444f-bb59-f5ebbbf4f32c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.404455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc4db-eefd-444f-bb59-f5ebbbf4f32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.406058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415bc4db-eefd-444f-bb59-f5ebbbf4f32c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.415815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1220/2455] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:25.420008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.430288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c477e2-af66-41a7-be9d-76ef0aa4ee4d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.435515 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.436426 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.436713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.436996 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] PATH_INFO: `/v3/projects/84f609a1cac14d03aad1b4f5a8e8306b/users/f069f64418dd43b2b7a644db1074b5d6/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.438979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f609a1cac14d03aad1b4f5a8e8306b, user_id=f069f64418dd43b2b7a644db1074b5d6, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.441697 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.452424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1236/2456] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:25 2022] PUT /identity/v3/projects/84f609a1cac14d03aad1b4f5a8e8306b/users/f069f64418dd43b2b7a644db1074b5d6/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:25.462337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.462628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.462908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.489232 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] MFA Rules not processed for user `f069f64418dd43b2b7a644db1074b5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:25.490410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:25.558166 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1221/2457] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:25 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:25.568534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:25.570318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:25.630938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552482bb-848f-481f-9353-74fb01a6405c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:25.634657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bb4c8-6b77-4b06-bda3-34a89cf15179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:25.635305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb4c8-6b77-4b06-bda3-34a89cf15179 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:25.635590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb4c8-6b77-4b06-bda3-34a89cf15179 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:25.635862 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bb4c8-6b77-4b06-bda3-34a89cf15179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:25.644868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bb4c8-6b77-4b06-bda3-34a89cf15179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:25.646304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bb4c8-6b77-4b06-bda3-34a89cf15179 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:25.655832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1237/2458] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:25 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:26.969801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:27.034118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c5b878-cc61-4bdc-94ee-53bd71edd9a9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:27.038608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:27.039468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:27.039750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:27.040044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] PATH_INFO: `/v3/users/382e8ab4d0f646cd93a7f9025a454e6a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:27.041224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=382e8ab4d0f646cd93a7f9025a454e6a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:27.043749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:27.092024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382e8ab4d0f646cd93a7f9025a454e6a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.099758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382e8ab4d0f646cd93a7f9025a454e6a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.100433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382e8ab4d0f646cd93a7f9025a454e6a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.101049 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382e8ab4d0f646cd93a7f9025a454e6a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.103159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:27.110742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382e8ab4d0f646cd93a7f9025a454e6a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.128784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1222/2459] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:26 2022] DELETE /identity/v3/users/382e8ab4d0f646cd93a7f9025a454e6a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:27.133202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:27.133679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:27.133871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:27.151927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:27.153167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:27.187118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:27.213357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1238/2460] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:00:27.222986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:27.224399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:27.281786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc6e70-833f-40f2-ae5c-fd4d25643057 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:27.285505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:27.286094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:27.286382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:27.286661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:27.296732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:27.298175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:27.310260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1223/2461] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:27.597704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:27.599696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:27.615818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83741f31-7702-44f9-8a38-b7c82f2f6105 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:27.624419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:27.625511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:27.625705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:27.626043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] PATH_INFO: `/v3/projects/3b6933e4365243d1b4a68abada8f6d83` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:27.627362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b6933e4365243d1b4a68abada8f6d83)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:27.629441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:27.711284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b6933e4365243d1b4a68abada8f6d83', 'request_id': 'req-95876aab-b5de-4548-b107-7d8e9735511c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.720569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b6933e4365243d1b4a68abada8f6d83', 'request_id': 'req-95876aab-b5de-4548-b107-7d8e9735511c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.727526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b6933e4365243d1b4a68abada8f6d83', 'request_id': 'req-95876aab-b5de-4548-b107-7d8e9735511c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.738652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] The token cache is being invalidate because project 3b6933e4365243d1b4a68abada8f6d83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:27.738954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.739615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.741277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1239/2462] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:27 2022] DELETE /identity/v3/projects/3b6933e4365243d1b4a68abada8f6d83 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:27.746247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:27.759665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9601ce-0389-4328-a397-8dfd81f6a3b3 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:27.828172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:27.829055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:27.829377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:27.829712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] PATH_INFO: `/v3/users/f069f64418dd43b2b7a644db1074b5d6` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:27.830865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f069f64418dd43b2b7a644db1074b5d6)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:27.833041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:27.880839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f069f64418dd43b2b7a644db1074b5d6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.888041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f069f64418dd43b2b7a644db1074b5d6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.888698 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f069f64418dd43b2b7a644db1074b5d6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.889909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f069f64418dd43b2b7a644db1074b5d6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.896983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ca66fa9e-ae72-401f-9a12-44e232194147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f069f64418dd43b2b7a644db1074b5d6'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:27.908012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:27.915823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1224/2463] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:27 2022] DELETE /identity/v3/users/f069f64418dd43b2b7a644db1074b5d6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:28.201989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.203922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:28.220719 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95876aab-b5de-4548-b107-7d8e9735511c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.293706 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.294829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.295253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.295593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] PATH_INFO: `/v3/projects/84f609a1cac14d03aad1b4f5a8e8306b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.296940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f609a1cac14d03aad1b4f5a8e8306b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.298911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.351774 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f609a1cac14d03aad1b4f5a8e8306b', 'request_id': 'req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:28.361198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f609a1cac14d03aad1b4f5a8e8306b', 'request_id': 'req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:28.368104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f609a1cac14d03aad1b4f5a8e8306b', 'request_id': 'req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:28.373764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:28.382157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] The token cache is being invalidate because project 84f609a1cac14d03aad1b4f5a8e8306b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:28.382730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:28.383696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cd55e355-c7bc-4103-b0eb-7c6b7897cad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:28.385260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1240/2464] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:28 2022] DELETE /identity/v3/projects/84f609a1cac14d03aad1b4f5a8e8306b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:28.390992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.391501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.391607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.409147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:28.410349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:28.469734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1225/2465] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:28.477452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.477747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.478031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.499051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:28.500296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:28.508308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1241/2466] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:28.513650 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.525847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da01725-f17c-4b81-b04e-895ff4a50428 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.531868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.532701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.533058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.533203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.534043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:28.534537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.536576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.552367 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1226/2467] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 20:00:28.556769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.567260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc4615-f20c-4a78-a9a4-4959c8f9de23 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.572606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.573275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.574255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.574580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.575294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.577298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.604532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1242/2468] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:28 2022] POST /identity/v3/projects => generated 407 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:28.609187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.621128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85d87be-5b9b-49a8-9dbd-ffaa308b79ae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.691200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.691887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.692209 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.692549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.693257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.695247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.715963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1227/2469] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:28 2022] POST /identity/v3/users => generated 304 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:28.720034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.731181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d7cc-66ce-42e0-b2d1-a495349264ec admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.735956 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.736658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.736939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.737222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.737944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.740111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.759754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1243/2470] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:28.761754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.775410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe12974-d40d-499f-bf8a-207aba8793ef admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.781155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4e8eb-4f6e-4664-ad9b-03ef3b456bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.781838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4e8eb-4f6e-4664-ad9b-03ef3b456bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.782128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4e8eb-4f6e-4664-ad9b-03ef3b456bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.782411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4e8eb-4f6e-4664-ad9b-03ef3b456bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.783156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4e8eb-4f6e-4664-ad9b-03ef3b456bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.785151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc4e8eb-4f6e-4664-ad9b-03ef3b456bc1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.798903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1228/2471] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:28.800337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:28.811434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41f34f-51f6-4363-acd5-5ef361900cd2 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:28.816010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:28.816933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.817215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.817492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] PATH_INFO: `/v3/projects/03a551470f2c4b4bac4eb19754ad8945/users/ec7d8099146d469d9fbd10460d4e6b8d/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.833667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a551470f2c4b4bac4eb19754ad8945, user_id=ec7d8099146d469d9fbd10460d4e6b8d, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:28.836377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:28.852400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1244/2472] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:28 2022] PUT /identity/v3/projects/03a551470f2c4b4bac4eb19754ad8945/users/ec7d8099146d469d9fbd10460d4e6b8d/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:28.864043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:28.864444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:28.865011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:28.895117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] MFA Rules not processed for user `ec7d8099146d469d9fbd10460d4e6b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:28.896419 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:28.954139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:28.970222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1229/2473] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:28 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:28.974891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.036877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a2f54-8576-4241-a8c3-bd764431bd4f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.041608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.042754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.043070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.043356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.044053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.046015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.071699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1245/2474] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:28 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:29.076732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.087269 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4601b5-78d5-486d-8dc7-75bb51eb4faa None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.142697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.143365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.143644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.143920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.144641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.146560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.168223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1230/2475] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:29 2022] POST /identity/v3/users => generated 311 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:29.171887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.185170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54ba02-d6fb-449c-abfa-296d8113a98d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.189973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.190761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.190894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.191164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.191885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.193532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.204825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1246/2476] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:29.208964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.219950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d1186-25ee-46c3-83f8-f9823086817b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.224792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.225667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.225961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.226221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] PATH_INFO: `/v3/projects/70317b9936d84d57bc5556bcb46ac51d/users/332ebf16c115407192be4c73a41b898a/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.241531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70317b9936d84d57bc5556bcb46ac51d, user_id=332ebf16c115407192be4c73a41b898a, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.244223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.257322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1231/2477] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:29 2022] PUT /identity/v3/projects/70317b9936d84d57bc5556bcb46ac51d/users/332ebf16c115407192be4c73a41b898a/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:29.261761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.271941 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f88765-b5ed-4a2c-9a97-ea09225e3bf9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.329015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e04b1f-a41e-492c-a392-28ba21666aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.329683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e04b1f-a41e-492c-a392-28ba21666aff admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.329970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e04b1f-a41e-492c-a392-28ba21666aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.330250 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e04b1f-a41e-492c-a392-28ba21666aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.330988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e04b1f-a41e-492c-a392-28ba21666aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.332624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e04b1f-a41e-492c-a392-28ba21666aff admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.337685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:29.344137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1247/2478] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:29.348147 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.358247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7278c-4620-4547-a096-1774cebfdcf6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.362947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.363867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.364205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.364513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] PATH_INFO: `/v3/projects/70317b9936d84d57bc5556bcb46ac51d/users/332ebf16c115407192be4c73a41b898a/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.366482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] RBAC: Authorizing `identity:create_grant(project_id=70317b9936d84d57bc5556bcb46ac51d, user_id=332ebf16c115407192be4c73a41b898a, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.369159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.379753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1232/2479] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:29 2022] PUT /identity/v3/projects/70317b9936d84d57bc5556bcb46ac51d/users/332ebf16c115407192be4c73a41b898a/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:29.390216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.390506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.390785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.418337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] MFA Rules not processed for user `332ebf16c115407192be4c73a41b898a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:29.419679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:29.494839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1248/2480] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:29 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:29.506977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.508792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:29.571849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18febcd-1b02-432e-944a-b2b638c6010d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.576062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.576743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.577025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.577299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.586679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.588137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.598723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1233/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:29 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:29.800971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:29.810884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b924b69b-0997-4341-aa9b-bdf47c6e73a1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:29.814724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba42ba8-3637-4e74-9e12-53625154c9ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:29.815457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42ba8-3637-4e74-9e12-53625154c9ce service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:29.815807 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42ba8-3637-4e74-9e12-53625154c9ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:29.816188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba42ba8-3637-4e74-9e12-53625154c9ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:29.872224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba42ba8-3637-4e74-9e12-53625154c9ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:29.873925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba42ba8-3637-4e74-9e12-53625154c9ce service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:29.882744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1249/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:29 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:35.174871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:35.176333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:35.225594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:35.242927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458361a-c1d6-4896-87fe-66d2d12dd5b1 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:35.248372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:35.249233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:35.249512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:35.249783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] PATH_INFO: `/v3/users/a1fe476c05784cf495602c9a06b355cf` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:35.250958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1fe476c05784cf495602c9a06b355cf)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:35.252933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:35.304878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1fe476c05784cf495602c9a06b355cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.312272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1fe476c05784cf495602c9a06b355cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.312958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1fe476c05784cf495602c9a06b355cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.313571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1fe476c05784cf495602c9a06b355cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.320454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1fe476c05784cf495602c9a06b355cf'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.338120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1234/2483] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:35 2022] DELETE /identity/v3/users/a1fe476c05784cf495602c9a06b355cf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:35.343120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:35.343655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:35.343655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:35.360622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:35.361751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:35.420050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1250/2484] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:35.429607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:35.431528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:35.490806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f16e58-e54a-421e-ad3a-30897df286d0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:35.494460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:35.495683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:35.495975 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:35.496285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:35.507042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:35.508532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:35.519041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1235/2485] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:35.795677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:35.797198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:35.809844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43393c36-57c6-42c6-a7d1-22d76ed8f62d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:35.815490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:35.816674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:35.816963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:35.817267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] PATH_INFO: `/v3/projects/2b6cf5094be9478ebd0c690849b7cd51` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:35.818522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b6cf5094be9478ebd0c690849b7cd51)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:35.821047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:35.844760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:35.870873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b6cf5094be9478ebd0c690849b7cd51', 'request_id': 'req-31f4198f-fc92-4204-bbc8-cb6f270630e9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.878716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b6cf5094be9478ebd0c690849b7cd51', 'request_id': 'req-31f4198f-fc92-4204-bbc8-cb6f270630e9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.885460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b6cf5094be9478ebd0c690849b7cd51', 'request_id': 'req-31f4198f-fc92-4204-bbc8-cb6f270630e9'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.895641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] The token cache is being invalidate because project 2b6cf5094be9478ebd0c690849b7cd51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:35.895923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.896593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:35.898133 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1251/2486] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:35 2022] DELETE /identity/v3/projects/2b6cf5094be9478ebd0c690849b7cd51 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:35.902652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:35.912443 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e477543-c055-460c-9b4e-e6181eb9a7e4 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:35.969125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:35.970120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:35.970429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:35.970730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] PATH_INFO: `/v3/users/9e4199bc39054df0bffcddcc2054c92c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:35.972055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e4199bc39054df0bffcddcc2054c92c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:35.974325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:35.985400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:36.024131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e4199bc39054df0bffcddcc2054c92c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.032984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e4199bc39054df0bffcddcc2054c92c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.034145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e4199bc39054df0bffcddcc2054c92c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.034812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e4199bc39054df0bffcddcc2054c92c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.043792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b01c6cea-bf31-4cd8-9882-60191ebcfe0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e4199bc39054df0bffcddcc2054c92c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.061292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1236/2487] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:35 2022] DELETE /identity/v3/users/9e4199bc39054df0bffcddcc2054c92c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:36.352483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.354061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:36.369509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4198f-fc92-4204-bbc8-cb6f270630e9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.429913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.430645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.430928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.431200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] PATH_INFO: `/v3/projects/54c73c4de5ac407586d174e2df804562` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.432438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c73c4de5ac407586d174e2df804562)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.434884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.481940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c73c4de5ac407586d174e2df804562', 'request_id': 'req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.489130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c73c4de5ac407586d174e2df804562', 'request_id': 'req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.495659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c73c4de5ac407586d174e2df804562', 'request_id': 'req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.507676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] The token cache is being invalidate because project 54c73c4de5ac407586d174e2df804562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:36.507962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.508718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-8b21b6f1-c61c-47eb-9d8b-3bb57698e302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:36.510256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1252/2488] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:36 2022] DELETE /identity/v3/projects/54c73c4de5ac407586d174e2df804562 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:36.515163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.515816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.516365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.540664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:36.543089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:36.618446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1237/2489] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:36.627137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.627499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.627912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.646576 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:36.647763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:36.656569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1253/2490] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:36.661159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.671816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3718af-04b2-4023-9a86-91f1a73d72dc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.676465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.677112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.677392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.677667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.678392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:36.678697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.680374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.692023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1238/2491] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:36.696338 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.705622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d75708-690e-4f1e-b86f-0031fce0ea7b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.710233 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.710878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.711161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.711437 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.712157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.714716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.741616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1254/2492] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:36 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:36.746915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.758913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91100e89-ac0e-49f8-ab27-28d9ff0e6584 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.818468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.819177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.819468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.819746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.820487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.822488 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.844246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1239/2493] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:36 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:36.848318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.858860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6eaeb-4670-4290-b153-05c30767ecb4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.863661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.864652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.864932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.865205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.865919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.867550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.869609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:36.880717 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1255/2494] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:36.884444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.891750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:36.895879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db91f4-928c-442f-98bf-8b1e318cd814 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.902711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af6eb8c-cc18-41c7-9270-0c49e464f30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.903582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6eb8c-cc18-41c7-9270-0c49e464f30b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.903930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6eb8c-cc18-41c7-9270-0c49e464f30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.904580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af6eb8c-cc18-41c7-9270-0c49e464f30b admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.905322 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6eb8c-cc18-41c7-9270-0c49e464f30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.906929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af6eb8c-cc18-41c7-9270-0c49e464f30b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.922724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1240/2495] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:36.923892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:36.934492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caa6793-22ac-4504-a342-024154511897 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:36.939033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:36.939911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.940231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.940511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] PATH_INFO: `/v3/projects/1987e5b58e8a4b9f9357456702ac2f3d/users/765b2da7cf2543bda5088fc7f22a9a9f/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:36.956842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1987e5b58e8a4b9f9357456702ac2f3d, user_id=765b2da7cf2543bda5088fc7f22a9a9f, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:36.959686 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:36.973335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1256/2496] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:36 2022] PUT /identity/v3/projects/1987e5b58e8a4b9f9357456702ac2f3d/users/765b2da7cf2543bda5088fc7f22a9a9f/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:36.985639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:36.985971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:36.986284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.014407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] MFA Rules not processed for user `765b2da7cf2543bda5088fc7f22a9a9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:37.015648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:37.080039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1241/2497] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:37.084738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.147642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be193d-b6dd-4f29-b146-f63e7e995d16 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.153160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.153818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.154104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.154384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.155073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.157061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.182388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1257/2498] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:37 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:37.186995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.197512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6f528-9559-4446-a24a-4a5fd08c3e3f None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.261889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.262561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.262852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.263130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.263829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.266205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.287632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1242/2499] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:37 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:37.291678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.302544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e062731-7474-46dc-b327-dc5497227a7f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.307081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.307766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.308174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.308468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.309197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.310821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.321747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1258/2500] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:37.325880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.336252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9959e0d-dd50-4f05-bf7a-7e45f1407a4e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.345264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.346163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.346449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.346738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] PATH_INFO: `/v3/projects/a9a75311d1634d30849f822d8a7b282d/users/aff8d7fc259848459752b3e0e5e666b8/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.362376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a75311d1634d30849f822d8a7b282d, user_id=aff8d7fc259848459752b3e0e5e666b8, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.365101 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.378185 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1243/2501] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:37 2022] PUT /identity/v3/projects/a9a75311d1634d30849f822d8a7b282d/users/aff8d7fc259848459752b3e0e5e666b8/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:37.382167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.393271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab511052-dae0-4bb2-8129-afc00db6104d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.451713 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c8bfd-8f7d-4607-b689-1f6f5aa893fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.452326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8bfd-8f7d-4607-b689-1f6f5aa893fa admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.452613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8bfd-8f7d-4607-b689-1f6f5aa893fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.452897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c8bfd-8f7d-4607-b689-1f6f5aa893fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.453620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c8bfd-8f7d-4607-b689-1f6f5aa893fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.455243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c8bfd-8f7d-4607-b689-1f6f5aa893fa admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.465712 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1259/2502] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:37.470330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.481776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668e9c63-37ce-4fc2-af78-d0731774a841 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.486450 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.487343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.487629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.487910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] PATH_INFO: `/v3/projects/a9a75311d1634d30849f822d8a7b282d/users/aff8d7fc259848459752b3e0e5e666b8/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.490084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a75311d1634d30849f822d8a7b282d, user_id=aff8d7fc259848459752b3e0e5e666b8, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.492822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.503452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1244/2503] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:37 2022] PUT /identity/v3/projects/a9a75311d1634d30849f822d8a7b282d/users/aff8d7fc259848459752b3e0e5e666b8/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:37.514398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.514691 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.515028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.541001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] MFA Rules not processed for user `aff8d7fc259848459752b3e0e5e666b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:37.543002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:37.575431 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:37.611493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1260/2504] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:37.625131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:37.627412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:37.686977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:37.707932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46224ca6-d09f-4af8-92f7-ee37c0741c6f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:37.711603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:37.712288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:37.712578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:37.712858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:37.786403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:37.787864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:37.798125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1245/2505] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:38.434402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:38.436413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:38.504152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9551c-3ec2-41b4-aa5e-663eb3dd6702 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:38.508184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:38.508986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:38.509322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:38.509653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:38.513454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:38.525157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:38.528220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4247c7-4c4f-40fc-8f1a-79930e18e839 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:38.538585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1261/2506] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:47.793189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=32 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:48.533739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=40 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:50.675670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:50.677140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:50.742705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d72b50-1aef-49bb-9b00-5730a0309276 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:50.748836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:50.749705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:50.749986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:50.750256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] PATH_INFO: `/v3/users/ec7d8099146d469d9fbd10460d4e6b8d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:50.751630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec7d8099146d469d9fbd10460d4e6b8d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:50.753603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:50.800582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec7d8099146d469d9fbd10460d4e6b8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:50.807633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec7d8099146d469d9fbd10460d4e6b8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:50.808283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec7d8099146d469d9fbd10460d4e6b8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:50.808899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec7d8099146d469d9fbd10460d4e6b8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:50.816822 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec7d8099146d469d9fbd10460d4e6b8d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:50.834540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1246/2507] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:50 2022] DELETE /identity/v3/users/ec7d8099146d469d9fbd10460d4e6b8d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:50.839283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:50.839750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:50.839971 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:50.857838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:50.859034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:50.917931 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1262/2508] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:50.927585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:50.928990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:50.988180 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117a7a6-3264-4cbd-a73e-a62896be0d2b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:50.991904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:50.992588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:50.992871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:50.993144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:51.002807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:51.004259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:51.013571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1247/2509] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:51.304776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:51.306644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:51.322085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db327351-37d6-41ad-aa87-5c32d82e1de9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:51.339333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:51.340363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:51.340863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:51.341775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] PATH_INFO: `/v3/projects/03a551470f2c4b4bac4eb19754ad8945` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:51.343284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a551470f2c4b4bac4eb19754ad8945)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:51.345569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:51.399041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a551470f2c4b4bac4eb19754ad8945', 'request_id': 'req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.407264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a551470f2c4b4bac4eb19754ad8945', 'request_id': 'req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.414800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a551470f2c4b4bac4eb19754ad8945', 'request_id': 'req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.425751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] The token cache is being invalidate because project 03a551470f2c4b4bac4eb19754ad8945 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:51.426046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.426723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.428352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1263/2510] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:51 2022] DELETE /identity/v3/projects/03a551470f2c4b4bac4eb19754ad8945 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:51.433465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:51.445554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddf4b8-2a15-4c9b-bbec-239f984ad20c service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:51.473939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:51.528447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:51.529663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:51.530183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:51.530714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:51.531146 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:51.531430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] PATH_INFO: `/v3/users/332ebf16c115407192be4c73a41b898a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:51.532683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] RBAC: Authorizing `identity:delete_user(user_id=332ebf16c115407192be4c73a41b898a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:51.534630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:51.586588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '332ebf16c115407192be4c73a41b898a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.596555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '332ebf16c115407192be4c73a41b898a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.597252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '332ebf16c115407192be4c73a41b898a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.597949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '332ebf16c115407192be4c73a41b898a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.602043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732a2ce-3a6f-4c5f-a5a9-87a4a925074a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:51.606089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '332ebf16c115407192be4c73a41b898a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:51.607480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:51.608173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:51.608462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:51.608790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:51.610688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:51.630080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1248/2511] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:51 2022] DELETE /identity/v3/users/332ebf16c115407192be4c73a41b898a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:00:51.672687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:51.714424 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:51.716201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f1b5c-d0da-4909-abe0-ab91d3ccf1a4 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:51.727463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1264/2512] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:00:51.997098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:51.999729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:52.102345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eede5f-be81-48fe-9937-187a30af822d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.107792 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.108757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.109041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.109318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] PATH_INFO: `/v3/projects/70317b9936d84d57bc5556bcb46ac51d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.110530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70317b9936d84d57bc5556bcb46ac51d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.112513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.166006 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:52.174234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70317b9936d84d57bc5556bcb46ac51d', 'request_id': 'req-eb5c4d5b-c366-4006-8a4f-d99976ffb073'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:52.184154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70317b9936d84d57bc5556bcb46ac51d', 'request_id': 'req-eb5c4d5b-c366-4006-8a4f-d99976ffb073'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:52.190225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70317b9936d84d57bc5556bcb46ac51d', 'request_id': 'req-eb5c4d5b-c366-4006-8a4f-d99976ffb073'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:52.205207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] The token cache is being invalidate because project 70317b9936d84d57bc5556bcb46ac51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:52.205498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:52.206117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-eb5c4d5b-c366-4006-8a4f-d99976ffb073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:52.207655 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1249/2513] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:51 2022] DELETE /identity/v3/projects/70317b9936d84d57bc5556bcb46ac51d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:52.212769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.213283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.213610 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.237709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:52.239051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:52.311715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1265/2514] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:52.313490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.313801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.314059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.332420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:52.333635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:52.347598 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1250/2515] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:00:52.349010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.360720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f82a95-d943-49be-b0ec-c824c4e8f4da None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.365627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.366298 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.366584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.366884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.367616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:52.367913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.369566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.382948 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1266/2516] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:52.387330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.397121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f5b2c-f4a1-466d-b2a7-6b024268f3bc None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.401943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.402607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.402893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.403174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.404503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.406715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.439415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1251/2517] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:52 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:52.442846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.454015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8930c9-f5b4-47eb-8c22-8b1d69979d1c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.526163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.526334 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.526516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.526788 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.527553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.530385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.546327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.548724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:52.556293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1267/2518] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:52 2022] POST /identity/v3/users => generated 284 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:52.560988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.573299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afb87db-4639-4544-9b77-37cb1b268b3b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.579611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.580622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.581202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.581668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.582580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.584455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.597835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1268/2519] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:52.602447 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.623478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69106e0e-9b28-4e07-98c0-fb1d51eb95bf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.628878 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.629653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.630015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.630387 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.631210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.632968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.646616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1269/2520] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:52.653200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.666561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadfc7d5-3a88-4b39-b660-b4d17b84983a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.673343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc30d-04a4-40b8-b395-140738134dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.674454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc30d-04a4-40b8-b395-140738134dec admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.674842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc30d-04a4-40b8-b395-140738134dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.675219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc30d-04a4-40b8-b395-140738134dec admin admin] PATH_INFO: `/v3/projects/1c2e63002cdc46f899c4f6030a4d3790/users/476d99f043c24515b22801f17bc4d2a9/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.682658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c90783-c9b3-429d-b9bb-89f5597e2152 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.692151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.692985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.693312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.693716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.696011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:52.697658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc30d-04a4-40b8-b395-140738134dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2e63002cdc46f899c4f6030a4d3790, user_id=476d99f043c24515b22801f17bc4d2a9, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.702139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc30d-04a4-40b8-b395-140738134dec admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.718809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.722706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:52.730768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1270/2521] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:52 2022] PUT /identity/v3/projects/1c2e63002cdc46f899c4f6030a4d3790/users/476d99f043c24515b22801f17bc4d2a9/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:52.732732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.733025 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.733306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.739781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1252/2522] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:00:52.746487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:52.780415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] MFA Rules not processed for user `476d99f043c24515b22801f17bc4d2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:52.782872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:52.877845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1271/2523] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:52.882933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:52.896581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbe13f-69f1-4147-ad67-92a3477c5040 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:52.968107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:52.968794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:52.969082 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:52.969360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:52.970070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:52.972017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.001764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1253/2524] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:52 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:53.006091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.017751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ae1b-bbad-4318-9e19-2607b677d25d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.081978 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.082671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.082985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.083265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.084011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.086033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.111481 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1272/2525] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:53 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:00:53.115542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.128376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f95993-caa9-4eac-9e9b-8b5e0c04ea55 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.133448 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.134241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.134637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.135016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.135902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.137675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.156383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1254/2526] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:53.161307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.177624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb994a-d5e5-475e-9f8b-fab0072ee9f1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.183676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.184865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.185668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.185668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] PATH_INFO: `/v3/projects/a44477c1718542b9a2674a18d8275c85/users/42b12456cd16424dae2fc13acb321c18/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.205413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44477c1718542b9a2674a18d8275c85, user_id=42b12456cd16424dae2fc13acb321c18, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.208191 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.229484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1273/2527] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:53 2022] PUT /identity/v3/projects/a44477c1718542b9a2674a18d8275c85/users/42b12456cd16424dae2fc13acb321c18/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:53.230034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.242832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:53.252435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab6e95-a031-4fef-86dc-18e109d6e596 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.360257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9ee6f-be4d-4b1a-b158-e7edf9c95c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.362241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ee6f-be4d-4b1a-b158-e7edf9c95c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.362764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ee6f-be4d-4b1a-b158-e7edf9c95c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.363356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9ee6f-be4d-4b1a-b158-e7edf9c95c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.365064 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9ee6f-be4d-4b1a-b158-e7edf9c95c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.369422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9ee6f-be4d-4b1a-b158-e7edf9c95c83 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.389031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1255/2528] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:00:53.393067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.413780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa32b9ce-bf0c-482c-a53e-8bb860fb8c45 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.428464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.429385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.429725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.430006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] PATH_INFO: `/v3/projects/a44477c1718542b9a2674a18d8275c85/users/42b12456cd16424dae2fc13acb321c18/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.440197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a44477c1718542b9a2674a18d8275c85, user_id=42b12456cd16424dae2fc13acb321c18, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.444751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.467983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1274/2529] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:53 2022] PUT /identity/v3/projects/a44477c1718542b9a2674a18d8275c85/users/42b12456cd16424dae2fc13acb321c18/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:53.474293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.475109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.475109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.509828 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] MFA Rules not processed for user `42b12456cd16424dae2fc13acb321c18`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:53.510916 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:53.604133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1256/2530] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:53.617265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.619393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:53.693377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:53.697760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcd9857-df9c-4301-8a3d-3d86b254aec0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.703454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.704207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.704497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.704920 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.715115 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.718679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:53.741372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1275/2531] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:53.893266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.905270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2105068-631d-4a63-99b2-e9226d54bb45 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:53.909863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890fe0bc-bd2b-4cdc-977b-58a2b5026eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:53.910111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fe0bc-bd2b-4cdc-977b-58a2b5026eaf service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:53.910412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fe0bc-bd2b-4cdc-977b-58a2b5026eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:53.910753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890fe0bc-bd2b-4cdc-977b-58a2b5026eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:53.985291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:53.986750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:53.988902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fe0bc-bd2b-4cdc-977b-58a2b5026eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:53.990433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890fe0bc-bd2b-4cdc-977b-58a2b5026eaf service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:54.001761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1257/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:53 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:54.078668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29690730-54a9-48e8-92b4-2d96275621b5 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:54.083728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:54.084851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:54.085142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:54.086063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] PATH_INFO: `/v3/users/bf62362e4f8c4901b4c9f5001f12a126` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:54.087332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf62362e4f8c4901b4c9f5001f12a126)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:54.089530 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:54.150448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf62362e4f8c4901b4c9f5001f12a126'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.159371 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf62362e4f8c4901b4c9f5001f12a126'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.160557 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf62362e4f8c4901b4c9f5001f12a126'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.161659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf62362e4f8c4901b4c9f5001f12a126'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.182748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf62362e4f8c4901b4c9f5001f12a126'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.198463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1276/2533] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:53 2022] DELETE /identity/v3/users/bf62362e4f8c4901b4c9f5001f12a126 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:54.204059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:54.205092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:54.205412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:54.224856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:54.226078 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:54.246527 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:54.319386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1258/2534] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:54.331396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:54.332995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:54.401907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be9e53-b292-49ef-87b2-328bc88d5cf3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:54.407480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ef60d-6329-4ad8-8e7a-65a94ea3e914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:54.408167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ef60d-6329-4ad8-8e7a-65a94ea3e914 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:54.408456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ef60d-6329-4ad8-8e7a-65a94ea3e914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:54.409720 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ef60d-6329-4ad8-8e7a-65a94ea3e914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:54.422108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ef60d-6329-4ad8-8e7a-65a94ea3e914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:54.423031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ef60d-6329-4ad8-8e7a-65a94ea3e914 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:54.436408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1277/2535] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:54.745799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:54.747942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:54.761974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801af24-d807-449f-b6d9-07a56b096aae None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:54.767326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:54.768292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:54.768576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:54.773641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] PATH_INFO: `/v3/projects/3049f8ee616f47088a44139a8ce036c1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:54.775981 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3049f8ee616f47088a44139a8ce036c1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:54.777982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:54.828714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3049f8ee616f47088a44139a8ce036c1', 'request_id': 'req-0db597cd-9e26-441d-971b-4f686768fc44'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.835033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3049f8ee616f47088a44139a8ce036c1', 'request_id': 'req-0db597cd-9e26-441d-971b-4f686768fc44'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.840653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3049f8ee616f47088a44139a8ce036c1', 'request_id': 'req-0db597cd-9e26-441d-971b-4f686768fc44'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.855735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] The token cache is being invalidate because project 3049f8ee616f47088a44139a8ce036c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:54.856128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.857315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0db597cd-9e26-441d-971b-4f686768fc44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:54.859641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1259/2536] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:54 2022] DELETE /identity/v3/projects/3049f8ee616f47088a44139a8ce036c1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:54.866198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:54.866489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:54.866799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:54.885255 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:54.886460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:54.892277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:54.948518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1278/2537] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:54.959008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:54.959352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:54.959633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:54.978394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:54.979588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:54.987523 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1260/2538] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:54.991593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.001800 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a264a5-f777-4507-90bd-3089a5bdca06 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.006414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.007067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.007353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.007631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.008397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:00:55.008707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.010324 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.022676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1279/2539] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:00:55.026898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.037054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c4ada0-574b-4466-9fc4-0fc09ad09401 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.041768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.042424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.042711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.042993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.043739 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.045687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.073957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1261/2540] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:00:55 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:55.078680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.089088 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5952314-00df-477a-bfe7-62e85d10535c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.146296 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.147002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.147295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.147565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.148344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.150316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.171558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1280/2541] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:00:55 2022] POST /identity/v3/users => generated 290 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:00:55.175767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.186617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d4fc4f-3b87-4a09-a481-1d448009624b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.191096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.191744 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.192027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.192341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.193065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.194660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.205283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1262/2542] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:55.209181 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.220251 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74908f4a-23f5-4a7b-b333-f106b2b46f6a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.225116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507e607-1760-433d-96b2-84065be6c11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.225914 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507e607-1760-433d-96b2-84065be6c11c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.226196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507e607-1760-433d-96b2-84065be6c11c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.226561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507e607-1760-433d-96b2-84065be6c11c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.227284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507e607-1760-433d-96b2-84065be6c11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.229101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8507e607-1760-433d-96b2-84065be6c11c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.240036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1281/2543] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:00:55.244175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.255339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047ed62-7252-4f9d-8c0e-7f33d2b42eda admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.260753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.261704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.262014 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.262331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] PATH_INFO: `/v3/projects/bc71f6e4fba640f2b41e5b3e4eb7f14d/users/0699a27504da472e96b2dbd52d6cf301/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.277193 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc71f6e4fba640f2b41e5b3e4eb7f14d, user_id=0699a27504da472e96b2dbd52d6cf301, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.280446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.293731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1263/2544] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:00:55 2022] PUT /identity/v3/projects/bc71f6e4fba640f2b41e5b3e4eb7f14d/users/0699a27504da472e96b2dbd52d6cf301/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:55.299558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.299854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.300187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.317817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:55.319011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:55.379295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1282/2545] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:00:55.389413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:55.391140 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:55.438196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:55.452142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581afaa1-339e-40ee-a665-aaf324d95300 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:55.456847 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211c67a-72c2-4dc0-a54b-ec0b41c052dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:55.457577 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c67a-72c2-4dc0-a54b-ec0b41c052dd service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:55.457933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c67a-72c2-4dc0-a54b-ec0b41c052dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:55.458274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211c67a-72c2-4dc0-a54b-ec0b41c052dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:55.468457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211c67a-72c2-4dc0-a54b-ec0b41c052dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:55.469905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211c67a-72c2-4dc0-a54b-ec0b41c052dd service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:55.479647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1264/2546] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:57.907643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:57.909560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:57.945829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7b4aa-3c79-4662-aaa3-9aeed7b9d37e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:57.968162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:57.969092 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:57.969373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:57.969646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] PATH_INFO: `/v3/users/765b2da7cf2543bda5088fc7f22a9a9f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:57.971772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=765b2da7cf2543bda5088fc7f22a9a9f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:57.973855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:57.998008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:58.062033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765b2da7cf2543bda5088fc7f22a9a9f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:58.088577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765b2da7cf2543bda5088fc7f22a9a9f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:58.089265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765b2da7cf2543bda5088fc7f22a9a9f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:58.090061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765b2da7cf2543bda5088fc7f22a9a9f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:58.116962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765b2da7cf2543bda5088fc7f22a9a9f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:58.149802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1283/2547] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:57 2022] DELETE /identity/v3/users/765b2da7cf2543bda5088fc7f22a9a9f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:58.155230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:58.199055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:58.199055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:58.218946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:58.218946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:58.293599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1265/2548] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:00:58.305860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:58.308630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:58.385977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffa15e0-178d-4f9e-b258-c2ecd6b9ff6f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:58.391890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:58.392987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:58.393417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:58.394428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:58.407831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:58.409539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:58.424966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1284/2549] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:00:58.914201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:58.916063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:58.931001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fea68-60cb-4e7e-903c-1258455eeb32 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:58.938355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:58.939513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:58.939995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:58.940434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] PATH_INFO: `/v3/projects/1987e5b58e8a4b9f9357456702ac2f3d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:58.941928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1987e5b58e8a4b9f9357456702ac2f3d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:58.945302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:59.009585 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1987e5b58e8a4b9f9357456702ac2f3d', 'request_id': 'req-a1542179-42df-41e8-b916-422f0624c22a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.016826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1987e5b58e8a4b9f9357456702ac2f3d', 'request_id': 'req-a1542179-42df-41e8-b916-422f0624c22a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.023942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1987e5b58e8a4b9f9357456702ac2f3d', 'request_id': 'req-a1542179-42df-41e8-b916-422f0624c22a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.046042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] The token cache is being invalidate because project 1987e5b58e8a4b9f9357456702ac2f3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:00:59.046596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.047806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.050094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1266/2550] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:58 2022] DELETE /identity/v3/projects/1987e5b58e8a4b9f9357456702ac2f3d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:59.054586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:59.075626 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65ef9-5730-456e-a85b-c04e4c878398 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:59.175449 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:59.175942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:59.176412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:59.176698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] PATH_INFO: `/v3/users/aff8d7fc259848459752b3e0e5e666b8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:59.178175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff8d7fc259848459752b3e0e5e666b8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:59.196831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:59.310779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:59.330352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff8d7fc259848459752b3e0e5e666b8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.358435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff8d7fc259848459752b3e0e5e666b8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.359806 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff8d7fc259848459752b3e0e5e666b8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.365205 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff8d7fc259848459752b3e0e5e666b8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.383008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-73020c8f-4529-40b0-bd53-71a989484f4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff8d7fc259848459752b3e0e5e666b8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.399227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1285/2551] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:59 2022] DELETE /identity/v3/users/aff8d7fc259848459752b3e0e5e666b8 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:59.463452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:59.467211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:59.484753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1542179-42df-41e8-b916-422f0624c22a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:00:59.619472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:00:59.620737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:59.621187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:59.621600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] PATH_INFO: `/v3/users/476d99f043c24515b22801f17bc4d2a9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:59.624122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=476d99f043c24515b22801f17bc4d2a9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:00:59.626287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:00:59.642657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:00:59.720795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '476d99f043c24515b22801f17bc4d2a9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.730401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '476d99f043c24515b22801f17bc4d2a9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.731211 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '476d99f043c24515b22801f17bc4d2a9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.732184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '476d99f043c24515b22801f17bc4d2a9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.741594 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '476d99f043c24515b22801f17bc4d2a9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:00:59.765223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1267/2552] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:00:59 2022] DELETE /identity/v3/users/476d99f043c24515b22801f17bc4d2a9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:00:59.769484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91879eec-e218-4612-a960-d49610c33c02 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:00:59.769988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91879eec-e218-4612-a960-d49610c33c02 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:00:59.770887 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91879eec-e218-4612-a960-d49610c33c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:00:59.793976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-91879eec-e218-4612-a960-d49610c33c02 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:00:59.795464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-91879eec-e218-4612-a960-d49610c33c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:59.893550 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:59.895343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:00:59.918672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1286/2553] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:00:59.930395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91879eec-e218-4612-a960-d49610c33c02 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:00:59.932322 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-91879eec-e218-4612-a960-d49610c33c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.023928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0046ed-02dc-4712-8c3d-ab7251f975c8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.043528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.045365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.045705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.045994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] PATH_INFO: `/v3/projects/a9a75311d1634d30849f822d8a7b282d` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.048298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a75311d1634d30849f822d8a7b282d)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.052941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.054160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91879eec-e218-4612-a960-d49610c33c02 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.058763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41c71-7e65-4a8e-832e-75c77c7583b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.061563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41c71-7e65-4a8e-832e-75c77c7583b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.062030 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41c71-7e65-4a8e-832e-75c77c7583b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.062484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41c71-7e65-4a8e-832e-75c77c7583b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.074523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41c71-7e65-4a8e-832e-75c77c7583b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.078444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41c71-7e65-4a8e-832e-75c77c7583b1 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.094900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1287/2554] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:01:00.130541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a75311d1634d30849f822d8a7b282d', 'request_id': 'req-a450ce1a-b29a-44a9-91b3-8279fd46dc93'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.141865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a75311d1634d30849f822d8a7b282d', 'request_id': 'req-a450ce1a-b29a-44a9-91b3-8279fd46dc93'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.152066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a75311d1634d30849f822d8a7b282d', 'request_id': 'req-a450ce1a-b29a-44a9-91b3-8279fd46dc93'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.197084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] The token cache is being invalidate because project a9a75311d1634d30849f822d8a7b282d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:00.197773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.200586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-a450ce1a-b29a-44a9-91b3-8279fd46dc93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.205967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1268/2555] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:00:59 2022] DELETE /identity/v3/projects/a9a75311d1634d30849f822d8a7b282d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:01:00.213574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.213861 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.214148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.261207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:00.263377 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.385995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:00.427042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1288/2556] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:00.429536 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.429831 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.430115 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.465142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:00.466503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.477858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1269/2557] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:00.481918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.492490 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828df8b-6568-4186-989e-ab40403df3f1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.500615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.501306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.501581 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.502151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.502935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:01:00.503245 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.505932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.543089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1289/2558] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:01:00.545084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.557950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05313dc-0548-44bc-bdc5-2f27ba5ba703 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.563591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bd267-98a5-4968-8a8a-f004e7855d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.564552 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bd267-98a5-4968-8a8a-f004e7855d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.564974 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bd267-98a5-4968-8a8a-f004e7855d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.565385 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bd267-98a5-4968-8a8a-f004e7855d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.566903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bd267-98a5-4968-8a8a-f004e7855d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.570067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1bd267-98a5-4968-8a8a-f004e7855d30 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.584499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:00.599139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.601863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.626453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1270/2559] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:00 2022] POST /identity/v3/projects => generated 375 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:00.629376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.629376 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.629684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.635182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8c26b-8421-4727-8aca-9f9506c11fc9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.667685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] MFA Rules not processed for user `0699a27504da472e96b2dbd52d6cf301`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:00.670700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.709440 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.710612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.710977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.711326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] PATH_INFO: `/v3/projects/1c2e63002cdc46f899c4f6030a4d3790` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.713280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2e63002cdc46f899c4f6030a4d3790)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.715301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.770431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1271/2560] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:00.771520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.785748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2e63002cdc46f899c4f6030a4d3790', 'request_id': 'req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.788065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657d5587-bc0c-4d72-b04e-6a51e5786405 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.795349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2e63002cdc46f899c4f6030a4d3790', 'request_id': 'req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.802838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2e63002cdc46f899c4f6030a4d3790', 'request_id': 'req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.815383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] The token cache is being invalidate because project 1c2e63002cdc46f899c4f6030a4d3790 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:00.815739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.816548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:00.818226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1290/2561] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:00 2022] DELETE /identity/v3/projects/1c2e63002cdc46f899c4f6030a4d3790 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:01:00.821895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.823353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.860277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.860966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.861257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.861546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.862362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.864467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.892427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1272/2562] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:00 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:00.892427 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.893875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:00.910683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb14f1-aa1d-4f4a-8371-6c01a0a59621 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.911941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522e734a-7c79-4f9e-b098-ece72bba0fe5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:00.918143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.919918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.922168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.922168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.922168 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.928606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.928693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:00.928693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:00.934414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:00.934414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] PATH_INFO: `/v3/users/42b12456cd16424dae2fc13acb321c18` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:00.934414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42b12456cd16424dae2fc13acb321c18)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:00.934414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:00.966960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:00.985321 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1291/2563] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:00 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:00.986767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:00.990456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.033449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42b12456cd16424dae2fc13acb321c18'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.043500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42b12456cd16424dae2fc13acb321c18'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.045268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42b12456cd16424dae2fc13acb321c18'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.045928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42b12456cd16424dae2fc13acb321c18'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.055083 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42b12456cd16424dae2fc13acb321c18'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.056324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:01.086494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d29913-c328-4cf3-9787-22f77d5c0a9b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.087445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1273/2564] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:01:00 2022] DELETE /identity/v3/users/42b12456cd16424dae2fc13acb321c18 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:01.088397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.089774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.178329 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.179132 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.179685 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.180069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.181090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.183567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.189769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb1569-e417-411e-aafb-d1444aae64f8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.196066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.196790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.197084 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.197364 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.198693 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.199472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1292/2565] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:01.203742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.204363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.206247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.224435 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036832f-fc87-4afa-acb4-c6fcc2c15593 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.235340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.236582 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.237215 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.240385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.240507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1274/2566] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:01 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:01.240592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.240592 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.241516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.255727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1293/2567] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:01.259562 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.261401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90d7b-4a48-4c20-8b50-e41ff558f63e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.267639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.268505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.268794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.269073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.270112 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.271750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.272684 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f505d0-aae2-49e1-a967-14fecb59828f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.279460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7f12e-893a-4c32-9cd3-f02892e87f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.280413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7f12e-893a-4c32-9cd3-f02892e87f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.280701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7f12e-893a-4c32-9cd3-f02892e87f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.281460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7f12e-893a-4c32-9cd3-f02892e87f1b admin admin] PATH_INFO: `/v3/projects/f90f9ace48b343d8b0048fe3ded0152f/users/50989df45a19444eab41710c5078fc57/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.303651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1275/2568] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:01.309035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.309641 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7f12e-893a-4c32-9cd3-f02892e87f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90f9ace48b343d8b0048fe3ded0152f, user_id=50989df45a19444eab41710c5078fc57, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.313058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7f12e-893a-4c32-9cd3-f02892e87f1b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.330191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8fd255-c3f5-4b8a-830e-0fc2bf858260 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.335728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1294/2569] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:01 2022] PUT /identity/v3/projects/f90f9ace48b343d8b0048fe3ded0152f/users/50989df45a19444eab41710c5078fc57/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:01:01.347478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4d759-1558-4e65-8137-86558148134d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.347922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4d759-1558-4e65-8137-86558148134d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.348022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4d759-1558-4e65-8137-86558148134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.383791 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-81b4d759-1558-4e65-8137-86558148134d None None] MFA Rules not processed for user `50989df45a19444eab41710c5078fc57`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:01.385198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-81b4d759-1558-4e65-8137-86558148134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.419526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.420603 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.421069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.421400 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] PATH_INFO: `/v3/projects/a2a6385911fa4d568b797dc90f3020b9/users/dd4ddc0930924161b67a5d428a327edb/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.445202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a6385911fa4d568b797dc90f3020b9, user_id=dd4ddc0930924161b67a5d428a327edb, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.448466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.482360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1276/2570] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:01 2022] PUT /identity/v3/projects/a2a6385911fa4d568b797dc90f3020b9/users/dd4ddc0930924161b67a5d428a327edb/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:01.484481 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.486317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.526966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:01.548589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1295/2571] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:01.548589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4d759-1558-4e65-8137-86558148134d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.549733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:01.562057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b4d759-1558-4e65-8137-86558148134d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.586868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb60a611-f74c-4852-8627-3bcd03213d66 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.594886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.595772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.596059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.596373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] PATH_INFO: `/v3/projects/a44477c1718542b9a2674a18d8275c85` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.597609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a44477c1718542b9a2674a18d8275c85)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.599559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.640880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.641668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.642096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.643084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.643084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.645052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.657164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1296/2572] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:01.661222 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.663613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.668516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a44477c1718542b9a2674a18d8275c85', 'request_id': 'req-c519befc-192c-482d-b1c5-07f3ef8bd369'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.681236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a44477c1718542b9a2674a18d8275c85', 'request_id': 'req-c519befc-192c-482d-b1c5-07f3ef8bd369'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.689832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a44477c1718542b9a2674a18d8275c85', 'request_id': 'req-c519befc-192c-482d-b1c5-07f3ef8bd369'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.704988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] The token cache is being invalidate because project a44477c1718542b9a2674a18d8275c85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:01.705564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.706393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:01.712464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1277/2573] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:01 2022] DELETE /identity/v3/projects/a44477c1718542b9a2674a18d8275c85 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:01.712464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:01.716107 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.778391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdbef70-ebf7-4f05-af84-356cdcf0a6b1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.808891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519befc-192c-482d-b1c5-07f3ef8bd369 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:01.815796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cb066-0104-439d-98dd-005d0c68dff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.816779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cb066-0104-439d-98dd-005d0c68dff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.818039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cb066-0104-439d-98dd-005d0c68dff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.818338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cb066-0104-439d-98dd-005d0c68dff8 admin admin] PATH_INFO: `/v3/projects/a2a6385911fa4d568b797dc90f3020b9/users/dd4ddc0930924161b67a5d428a327edb/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.821106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cb066-0104-439d-98dd-005d0c68dff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a6385911fa4d568b797dc90f3020b9, user_id=dd4ddc0930924161b67a5d428a327edb, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.823805 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2cb066-0104-439d-98dd-005d0c68dff8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.839465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1278/2574] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:01 2022] PUT /identity/v3/projects/a2a6385911fa4d568b797dc90f3020b9/users/dd4ddc0930924161b67a5d428a327edb/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:01.839465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfcf9e-5272-42f6-b12d-7eb9a32db515 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.839465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfcf9e-5272-42f6-b12d-7eb9a32db515 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.839465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabfcf9e-5272-42f6-b12d-7eb9a32db515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.850291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:01.851091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.851494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.851847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.854100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.864688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-eabfcf9e-5272-42f6-b12d-7eb9a32db515 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:01.866025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-eabfcf9e-5272-42f6-b12d-7eb9a32db515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.915694 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:01.939117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:01.940782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:01.952474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1279/2575] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:01.952821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1297/2576] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:01.960533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:01.961011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:01.961435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:01.975531 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:01.982755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:01.984161 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:01.996614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1280/2577] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:02.002456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.017259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0774fb39-a842-4d4d-810c-290487f4ec83 service glance] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.031463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.032851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.033419 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.033820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.035284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:01:02.035729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.037600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.056128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1298/2578] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 20:01:02.060854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.088462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bb056-de85-4335-984b-523a507519e6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.095155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.095876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.096719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.097009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.097772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.100301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.141947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1281/2579] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:02 2022] POST /identity/v3/projects => generated 357 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:02.146620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.158954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5791c0-d673-4833-8aec-50055ef7a46b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.265221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.266312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.266655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.266859 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.267596 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.270668 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.297998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1299/2580] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:02 2022] POST /identity/v3/users => generated 279 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:02.302335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.315445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f3080-747a-4116-8280-fa136252fecc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.321708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.323099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.323521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.323958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.325618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.329264 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.354198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1282/2581] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:02.355702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.371577 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e25f717-d910-45f1-a07c-76d48ae74cfc admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.382184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19bde4-c93d-4d42-820b-f1e8ed716a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.382972 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19bde4-c93d-4d42-820b-f1e8ed716a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.383258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19bde4-c93d-4d42-820b-f1e8ed716a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.386575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19bde4-c93d-4d42-820b-f1e8ed716a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.388902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19bde4-c93d-4d42-820b-f1e8ed716a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.390688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19bde4-c93d-4d42-820b-f1e8ed716a5f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.407049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1300/2582] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:02.411505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.426102 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5176669-63e0-4a6a-bd21-755a3a6ce966 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.431870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.433279 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.433707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.434149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] PATH_INFO: `/v3/projects/41e9df8d087e452597ebfcaebe6e69d8/users/993bc587ba2f41bb99e31efa8dc037b7/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.459492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41e9df8d087e452597ebfcaebe6e69d8, user_id=993bc587ba2f41bb99e31efa8dc037b7, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.462659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.488908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1283/2583] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:02 2022] PUT /identity/v3/projects/41e9df8d087e452597ebfcaebe6e69d8/users/993bc587ba2f41bb99e31efa8dc037b7/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:02.500163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.500459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.500786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.565098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] MFA Rules not processed for user `993bc587ba2f41bb99e31efa8dc037b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:02.566727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:02.708101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1301/2584] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:02 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:02.741402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.743366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:02.833249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc82d0-e1da-4336-9097-55048a6eb111 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.838993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5149bdb-06c2-4885-834c-09e5b6326364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.839850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5149bdb-06c2-4885-834c-09e5b6326364 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.840244 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5149bdb-06c2-4885-834c-09e5b6326364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.841598 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5149bdb-06c2-4885-834c-09e5b6326364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.853499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5149bdb-06c2-4885-834c-09e5b6326364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.855201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5149bdb-06c2-4885-834c-09e5b6326364 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:02.881310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1284/2585] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:02 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:02.890291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:02.940757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:02.975349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663842a-389e-4d90-a01b-2aba7df7c9c7 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:02.980553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:02.982272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:02.983154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:02.983539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] PATH_INFO: `/v3/users/993bc587ba2f41bb99e31efa8dc037b7` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:02.985525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=993bc587ba2f41bb99e31efa8dc037b7)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:02.987819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:03.078534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993bc587ba2f41bb99e31efa8dc037b7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:03.098011 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993bc587ba2f41bb99e31efa8dc037b7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:03.102065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993bc587ba2f41bb99e31efa8dc037b7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:03.109116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993bc587ba2f41bb99e31efa8dc037b7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:03.145145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993bc587ba2f41bb99e31efa8dc037b7'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:03.161062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1302/2586] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:01:02 2022] DELETE /identity/v3/users/993bc587ba2f41bb99e31efa8dc037b7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:03.165272 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:03.165666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:03.166041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:03.211588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:03.211588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:03.325663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1285/2587] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:03.332237 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:03.333677 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:03.424848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d89fc3-c99a-40b7-b839-3e21c68bacfa admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:03.439531 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add4157-1123-4ea5-b100-c6a0660e37d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:03.440267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add4157-1123-4ea5-b100-c6a0660e37d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:03.440598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add4157-1123-4ea5-b100-c6a0660e37d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:03.440815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add4157-1123-4ea5-b100-c6a0660e37d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:03.459786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add4157-1123-4ea5-b100-c6a0660e37d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:03.461487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add4157-1123-4ea5-b100-c6a0660e37d3 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:03.478521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1303/2588] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:04.489476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:04.490986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:04.504908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531ba1e-d7d1-47dc-9b29-f082e7a8dfbd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:04.509902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:04.510778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:04.511066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:04.511348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] PATH_INFO: `/v3/projects/41e9df8d087e452597ebfcaebe6e69d8` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:04.513745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41e9df8d087e452597ebfcaebe6e69d8)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:04.515788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:04.588810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:04.599288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41e9df8d087e452597ebfcaebe6e69d8', 'request_id': 'req-dac2f740-2107-4c1d-8eca-8526db0533d0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:04.620258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41e9df8d087e452597ebfcaebe6e69d8', 'request_id': 'req-dac2f740-2107-4c1d-8eca-8526db0533d0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:04.641290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41e9df8d087e452597ebfcaebe6e69d8', 'request_id': 'req-dac2f740-2107-4c1d-8eca-8526db0533d0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:04.675490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] The token cache is being invalidate because project 41e9df8d087e452597ebfcaebe6e69d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:04.675812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:04.677520 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-dac2f740-2107-4c1d-8eca-8526db0533d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:04.680142 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1286/2589] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:04 2022] DELETE /identity/v3/projects/41e9df8d087e452597ebfcaebe6e69d8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:04.686320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:04.687203 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:04.687635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:04.710184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:04.712288 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:04.785725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:04.819929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1304/2590] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:04.837198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:04.837501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:04.837790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:04.885237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:04.887042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:04.898196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1287/2591] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:04.902721 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:04.915681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb7ac1-e951-4934-9e54-c9d06ae42690 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:04.923229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:04.924292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:04.924723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:04.925184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:04.926137 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:01:04.926604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:04.928486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:04.945293 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1305/2592] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:01:04.950121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:04.965177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae549b-6f69-48af-aaef-72ae3191eb2a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:04.972266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:04.972969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:04.973258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:04.973609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:04.974328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:04.977507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:05.033275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1288/2593] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:04 2022] POST /identity/v3/projects => generated 373 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:05.035412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:05.052766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e5ddf-e0ff-41ee-92c1-2ae33f05c6d1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:05.158969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:05.159670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:05.159953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:05.160261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:05.160963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:05.162944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:05.227841 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1306/2594] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:05 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:01:05.240167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:05.254265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1efdf4-263e-4080-9ff9-c5301b03d223 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:05.261631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:05.263444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:05.263444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:05.263444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:05.264707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:05.265798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:05.278664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1289/2595] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:05.283256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:05.298710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa91c9f-20b5-4d93-b874-ca2387cd615d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:05.310148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45052f3-3a88-4028-8e18-d78121d3018f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:05.310960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45052f3-3a88-4028-8e18-d78121d3018f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:05.311918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45052f3-3a88-4028-8e18-d78121d3018f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:05.312306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45052f3-3a88-4028-8e18-d78121d3018f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:05.314767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45052f3-3a88-4028-8e18-d78121d3018f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:05.317588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45052f3-3a88-4028-8e18-d78121d3018f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:05.333946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1307/2596] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:01:05.338033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:05.359769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e42180-45f6-4f37-b9ee-69daefa8dbaf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:05.370309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:05.371241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:05.371521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:05.372678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] PATH_INFO: `/v3/projects/00872ac8b2f745688edb34094d77c11b/users/8311fc70c9a346e7b82d6bb700c2ace4/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:05.407635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00872ac8b2f745688edb34094d77c11b, user_id=8311fc70c9a346e7b82d6bb700c2ace4, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:05.410411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:05.447044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1290/2597] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:05 2022] PUT /identity/v3/projects/00872ac8b2f745688edb34094d77c11b/users/8311fc70c9a346e7b82d6bb700c2ace4/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:05.451553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:05.451856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:05.452180 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:05.489402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] MFA Rules not processed for user `8311fc70c9a346e7b82d6bb700c2ace4`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:05.491370 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:05.674213 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1308/2598] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:05.680301 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:05.832410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5e16e-4f58-4c52-bf24-3e134f84ff66 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:05.838770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:05.844348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:05.844776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:05.845229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:05.846231 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:05.848584 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:05.900271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1291/2599] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:05 2022] POST /identity/v3/projects => generated 373 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:05.905740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:05.924982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55cd916-23d3-45fc-b717-1a5ed3f13980 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:06.010909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:06.042356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:06.043042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.044482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.044798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.045615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:06.047747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:06.075113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1309/2600] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:05 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:01:06.079409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:06.112838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99220e9-b3ab-4a81-b147-7937f45bfcef admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:06.120666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:06.121493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.122999 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.123566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.125044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:06.126968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:06.143473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1292/2601] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:06.147445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:06.165410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a89d9-1731-4b59-ab13-69f09368ac3a admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:06.170902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:06.171813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.172108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.172396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] PATH_INFO: `/v3/projects/403b520051f04ebdb636c14e8b2bcc17/users/38679b05ed28451eadb4623f4f5356e1/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.192777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403b520051f04ebdb636c14e8b2bcc17, user_id=38679b05ed28451eadb4623f4f5356e1, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:06.196287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:06.216823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1310/2602] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:06 2022] PUT /identity/v3/projects/403b520051f04ebdb636c14e8b2bcc17/users/38679b05ed28451eadb4623f4f5356e1/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:06.222121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:06.254931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d74522-7fd6-407d-97a0-1dc0da6a1f8c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:06.313144 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:06.343182 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f311a213-c39e-4111-b469-9e96f906288a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:06.343781 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a213-c39e-4111-b469-9e96f906288a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.344063 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a213-c39e-4111-b469-9e96f906288a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.344372 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f311a213-c39e-4111-b469-9e96f906288a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.345096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311a213-c39e-4111-b469-9e96f906288a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:06.347398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f311a213-c39e-4111-b469-9e96f906288a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:06.363410 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1293/2603] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:06.365076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:06.391500 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a0a45-cec9-4dd0-b553-433b5620fa64 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:06.405379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:06.406156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.406738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.406874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] PATH_INFO: `/v3/projects/403b520051f04ebdb636c14e8b2bcc17/users/38679b05ed28451eadb4623f4f5356e1/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.410021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=403b520051f04ebdb636c14e8b2bcc17, user_id=38679b05ed28451eadb4623f4f5356e1, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:06.413748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:06.431140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1311/2604] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:06 2022] PUT /identity/v3/projects/403b520051f04ebdb636c14e8b2bcc17/users/38679b05ed28451eadb4623f4f5356e1/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:06.445880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef688-d74f-4fa0-9fc7-e5b2fc04b56e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.446319 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef688-d74f-4fa0-9fc7-e5b2fc04b56e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.446644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ef688-d74f-4fa0-9fc7-e5b2fc04b56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.507433 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-9c2ef688-d74f-4fa0-9fc7-e5b2fc04b56e None None] MFA Rules not processed for user `38679b05ed28451eadb4623f4f5356e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:06.508807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9c2ef688-d74f-4fa0-9fc7-e5b2fc04b56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:06.647429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1294/2605] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:06.661757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:06.664134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:06.747743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f1a2a-4ba2-4b5d-9ce4-e399b60619c5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:06.752825 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:06.754010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:06.754138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:06.754635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:06.842561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:06.844223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:06.857417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1312/2606] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:07.689098 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:07.689617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:07.689712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:07.737712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] MFA Rules not processed for user `dd4ddc0930924161b67a5d428a327edb`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:07.741977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:07.871135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1295/2607] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:07.888446 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:07.890568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:07.948950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:07.978829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca7dfe-2ee8-43ae-aa85-2a840bfa3785 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:07.986200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:07.990308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:07.990308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:07.990308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:07.992054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:08.062161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:08.063960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cd308-624a-4590-b9ec-7c5413a68c0e service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:08.077022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1313/2608] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:01:09.671409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:09.787705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:09.816681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dd4f11-7aac-4b77-949b-17d130c2db24 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:09.822543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:09.823586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:09.823874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:09.824249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] PATH_INFO: `/v3/users/8311fc70c9a346e7b82d6bb700c2ace4` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:09.826732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8311fc70c9a346e7b82d6bb700c2ace4)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:09.833159 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:09.894993 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8311fc70c9a346e7b82d6bb700c2ace4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:09.903649 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8311fc70c9a346e7b82d6bb700c2ace4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:09.904479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8311fc70c9a346e7b82d6bb700c2ace4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:09.905095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8311fc70c9a346e7b82d6bb700c2ace4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:09.912429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8311fc70c9a346e7b82d6bb700c2ace4'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:09.929865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1296/2609] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:01:09 2022] DELETE /identity/v3/users/8311fc70c9a346e7b82d6bb700c2ace4 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:09.934506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:09.934955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:09.935160 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:09.953235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:09.954465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:10.018518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1314/2610] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:10.029807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:10.031565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:10.111453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be1f9-61ed-4814-b1ed-b43efd5bb7b7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:10.115154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:10.115810 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:10.116106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:10.116397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:10.126991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:10.128641 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:10.138803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1297/2611] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:10.457391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:10.458899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:10.472029 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab8d92-0653-438b-a127-d0e038a0eb60 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:10.477805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:10.478756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:10.479144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:10.479489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] PATH_INFO: `/v3/projects/00872ac8b2f745688edb34094d77c11b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:10.481519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00872ac8b2f745688edb34094d77c11b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:10.483518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:10.540739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00872ac8b2f745688edb34094d77c11b', 'request_id': 'req-c08e4cd4-a935-4967-acef-fcf2b782e80c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.549002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00872ac8b2f745688edb34094d77c11b', 'request_id': 'req-c08e4cd4-a935-4967-acef-fcf2b782e80c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.556342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00872ac8b2f745688edb34094d77c11b', 'request_id': 'req-c08e4cd4-a935-4967-acef-fcf2b782e80c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.572728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] The token cache is being invalidate because project 00872ac8b2f745688edb34094d77c11b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:10.573021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.573687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.576229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1315/2612] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:10 2022] DELETE /identity/v3/projects/00872ac8b2f745688edb34094d77c11b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:10.580362 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:10.591681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd7869c-941c-49b3-ba32-de401a8bcb71 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:10.658250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:10.659163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:10.659444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:10.659745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] PATH_INFO: `/v3/users/38679b05ed28451eadb4623f4f5356e1` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:10.661353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38679b05ed28451eadb4623f4f5356e1)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:10.663274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:10.678093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:10.718278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38679b05ed28451eadb4623f4f5356e1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.726283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38679b05ed28451eadb4623f4f5356e1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.726984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38679b05ed28451eadb4623f4f5356e1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.727591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38679b05ed28451eadb4623f4f5356e1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.749800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4df305d5-f7ab-4a17-85fd-3abccf0e2a61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38679b05ed28451eadb4623f4f5356e1'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:10.771297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1298/2613] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:01:10 2022] DELETE /identity/v3/users/38679b05ed28451eadb4623f4f5356e1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:11.106058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.107856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:11.147656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e4cd4-a935-4967-acef-fcf2b782e80c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.162628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:11.260058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.261055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.261502 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.261853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] PATH_INFO: `/v3/projects/403b520051f04ebdb636c14e8b2bcc17` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.264422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=403b520051f04ebdb636c14e8b2bcc17)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:11.266733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:11.352032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '403b520051f04ebdb636c14e8b2bcc17', 'request_id': 'req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:11.374842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '403b520051f04ebdb636c14e8b2bcc17', 'request_id': 'req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:11.383689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '403b520051f04ebdb636c14e8b2bcc17', 'request_id': 'req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:11.398683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] The token cache is being invalidate because project 403b520051f04ebdb636c14e8b2bcc17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:11.399065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:11.399818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-5ea77e50-b2ab-4b69-aeb5-3389eddb71ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:11.401622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1316/2614] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:11 2022] DELETE /identity/v3/projects/403b520051f04ebdb636c14e8b2bcc17 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:11.407719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.408055 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.408369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.429208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:11.430475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:11.529762 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1299/2615] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:11.539802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.540276 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.540663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.566353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:11.567729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:11.582067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1317/2616] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:11.586827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.603005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd7cab-ddc6-4340-879a-9b2ea7c46f68 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.620109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.620109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.620109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.620109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.620825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:01:11.620825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:11.620825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:11.657845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1300/2617] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:01:11.662138 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.674073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34db6d8-5b9d-48d9-ac9e-b9d1735109ee None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.689366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.690086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.690534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.690845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.691642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:11.693621 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:11.737095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1318/2618] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:11 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:01:11.741963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.753461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4be55-4ab7-4d6c-9138-430a0e4bff59 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.831767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.832506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.832857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.833229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.833994 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:11.836327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:11.869748 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1301/2619] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:11 2022] POST /identity/v3/users => generated 286 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:11.870185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.883729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabcaa2-97cd-4e35-9140-ba881edd9451 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.890795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.891648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.892000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.892455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.893514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:11.897144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:11.916373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.922801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1319/2620] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:01:11.925171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:11.929538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2181c8f2-3aa1-4958-b333-b4aa143fdd6d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.941360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015bc68-2593-44e1-948c-0005b256314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.942058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015bc68-2593-44e1-948c-0005b256314e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.942330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015bc68-2593-44e1-948c-0005b256314e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.942609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015bc68-2593-44e1-948c-0005b256314e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:11.943354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015bc68-2593-44e1-948c-0005b256314e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:11.945002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015bc68-2593-44e1-948c-0005b256314e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:11.959534 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1302/2621] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:01:11.964122 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:11.975468 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52720a-c265-4d29-b132-dc3080096298 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:11.980210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:11.981766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:11.982042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:11.982343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] PATH_INFO: `/v3/projects/2a62977afd3c4b17ac43429d7e593580/users/24e3ac2f7c00476d8bcfb88aeb17293a/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:12.005507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a62977afd3c4b17ac43429d7e593580, user_id=24e3ac2f7c00476d8bcfb88aeb17293a, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:12.010235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:12.031631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1320/2622] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:11 2022] PUT /identity/v3/projects/2a62977afd3c4b17ac43429d7e593580/users/24e3ac2f7c00476d8bcfb88aeb17293a/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:12.037809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1b5a-9b02-4e47-bb72-d5636e318b21 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:12.038220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1b5a-9b02-4e47-bb72-d5636e318b21 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:12.038871 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c1b5a-9b02-4e47-bb72-d5636e318b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:12.075226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-db1c1b5a-9b02-4e47-bb72-d5636e318b21 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:12.076504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-db1c1b5a-9b02-4e47-bb72-d5636e318b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:12.158130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1303/2623] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:12.165220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:12.167302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:12.230422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:12.249873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cab640-90e2-430d-99ac-d519f0f3a994 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:12.253870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:12.254569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:12.255123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:12.255420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:12.265776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:12.267234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:12.278409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1321/2624] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:16.874684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:16.875202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:16.875291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:16.911073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] MFA Rules not processed for user `24e3ac2f7c00476d8bcfb88aeb17293a`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:16.912807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:17.002288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1304/2625] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:17.008975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:17.021802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197f774-c5f4-4b48-bae2-aead9ea6914a service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:17.032317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:17.033177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:17.033620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:17.034077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:17.034980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:17.037152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:17.082454 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1322/2626] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:17 2022] POST /identity/v3/projects => generated 371 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:01:17.083675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:17.095592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb87fbe1-dd42-4783-a03f-7394d0e8b24e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:17.167833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:17.169350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:17.169637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:17.169919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:17.170622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:17.172604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:17.195695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1305/2627] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:17 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:17.200930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:17.219731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b9dbbe-13ab-4528-ba95-7c122c57565c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:17.228070 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:17.228814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:17.229095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:17.229382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:17.230130 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:17.233145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:17.261084 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1323/2628] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:17.266106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:17.277099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc86664-b569-4685-8615-2263d7ad2fa7 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:17.282825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:17.283968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:17.284435 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:17.284719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] PATH_INFO: `/v3/projects/e27ff170237048be9dc5a7a3bc58dd3b/users/052b9567cd724e3291561862ea503d52/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:17.299850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:17.304464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27ff170237048be9dc5a7a3bc58dd3b, user_id=052b9567cd724e3291561862ea503d52, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:17.307750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:17.322246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1306/2629] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:17 2022] PUT /identity/v3/projects/e27ff170237048be9dc5a7a3bc58dd3b/users/052b9567cd724e3291561862ea503d52/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:01:17.328986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:17.350759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92aa938-5fc9-420c-a678-d1db53454d81 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:17.413835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b45397e-fd7f-45eb-b22c-6631602e47a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:17.414430 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45397e-fd7f-45eb-b22c-6631602e47a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:17.414715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45397e-fd7f-45eb-b22c-6631602e47a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:17.415002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b45397e-fd7f-45eb-b22c-6631602e47a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:17.415761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45397e-fd7f-45eb-b22c-6631602e47a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:17.418170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b45397e-fd7f-45eb-b22c-6631602e47a2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:17.430487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1324/2630] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:17.434845 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:17.445858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de0a57-0969-404c-a706-2fc9ce2a1d8a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:17.451669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:17.452637 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:17.452922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:17.453203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] PATH_INFO: `/v3/projects/e27ff170237048be9dc5a7a3bc58dd3b/users/052b9567cd724e3291561862ea503d52/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:17.455418 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e27ff170237048be9dc5a7a3bc58dd3b, user_id=052b9567cd724e3291561862ea503d52, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:17.458300 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:17.470422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1307/2631] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:17 2022] PUT /identity/v3/projects/e27ff170237048be9dc5a7a3bc58dd3b/users/052b9567cd724e3291561862ea503d52/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:22.309453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c549fc-93c6-4949-977d-2804ccfdb6e1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:22.309977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c549fc-93c6-4949-977d-2804ccfdb6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:22.309977 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c549fc-93c6-4949-977d-2804ccfdb6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:22.348716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-85c549fc-93c6-4949-977d-2804ccfdb6e1 None None] MFA Rules not processed for user `052b9567cd724e3291561862ea503d52`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:22.350196 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-85c549fc-93c6-4949-977d-2804ccfdb6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:22.442332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1325/2632] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:22.459392 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:22.461267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:22.552030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a38385-aa95-40b9-8624-f57bd45081c6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:22.558252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:22.559123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:22.559547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:22.559833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:22.671019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:22.680669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:22.695309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1308/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:24.301225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f03ded-e904-4760-a413-a3ceebbbc993 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:24.301835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f03ded-e904-4760-a413-a3ceebbbc993 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:24.301835 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f03ded-e904-4760-a413-a3ceebbbc993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:24.321662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c0f03ded-e904-4760-a413-a3ceebbbc993 None None] MFA Rules not processed for user `24e3ac2f7c00476d8bcfb88aeb17293a`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:24.322901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0f03ded-e904-4760-a413-a3ceebbbc993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:24.338060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1326/2634] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:32.688962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=65,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:32.869088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:32.870624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:32.949114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e853a-8f72-4158-b59c-bfa190edf1b6 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:32.954674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:32.955230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:32.955581 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:32.955928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] PATH_INFO: `/v3/users/50989df45a19444eab41710c5078fc57` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:32.957314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50989df45a19444eab41710c5078fc57)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:32.959333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:33.008917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50989df45a19444eab41710c5078fc57'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.017121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50989df45a19444eab41710c5078fc57'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.017735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50989df45a19444eab41710c5078fc57'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.018415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50989df45a19444eab41710c5078fc57'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.026998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50989df45a19444eab41710c5078fc57'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.045309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1309/2635] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:01:32 2022] DELETE /identity/v3/users/50989df45a19444eab41710c5078fc57 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:33.050128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.050580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.050784 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.068522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:33.069737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:33.087749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:33.140583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1327/2636] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:33.151174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:33.152755 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:33.218008 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209454b2-8ab3-46a9-87ed-e523e86ef380 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:33.224944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc1819-93a9-47a2-89ce-709fa6bd8ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:33.225587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc1819-93a9-47a2-89ce-709fa6bd8ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.225870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc1819-93a9-47a2-89ce-709fa6bd8ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.226143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc1819-93a9-47a2-89ce-709fa6bd8ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.237393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc1819-93a9-47a2-89ce-709fa6bd8ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:33.238788 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc1819-93a9-47a2-89ce-709fa6bd8ffe service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:33.250280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1310/2637] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:33.545810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:33.547297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:33.561202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec97d92-e0ae-4780-8b19-35f7c9161b74 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:33.565854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:33.566725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.567007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.567277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] PATH_INFO: `/v3/projects/f90f9ace48b343d8b0048fe3ded0152f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.568524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90f9ace48b343d8b0048fe3ded0152f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:33.570453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:33.620647 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90f9ace48b343d8b0048fe3ded0152f', 'request_id': 'req-4e011783-b529-40c7-9e8d-ae0eba3c8018'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.628529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90f9ace48b343d8b0048fe3ded0152f', 'request_id': 'req-4e011783-b529-40c7-9e8d-ae0eba3c8018'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.635333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90f9ace48b343d8b0048fe3ded0152f', 'request_id': 'req-4e011783-b529-40c7-9e8d-ae0eba3c8018'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.647716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] The token cache is being invalidate because project f90f9ace48b343d8b0048fe3ded0152f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:33.648028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.648802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4e011783-b529-40c7-9e8d-ae0eba3c8018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:33.650483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1328/2638] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:33 2022] DELETE /identity/v3/projects/f90f9ace48b343d8b0048fe3ded0152f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:33.656018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.656444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.656887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.677069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:33.678273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:33.697169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:33.752938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1311/2639] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:33.758768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.759060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.759343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.778798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:33.780151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:33.788178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1329/2640] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:33.792806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:33.803862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c6d6b8-733d-4fa2-afba-f4830b81d234 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:33.808719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:33.809401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.809671 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.809951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.811878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:01:33.812226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:33.813879 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:33.830951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1312/2641] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 20:01:33.832744 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:33.843243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfc8ff-86b4-4bd5-bbb0-91b635d5718f None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:33.848593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:33.849262 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.849541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.849814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.850518 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:33.852457 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:33.885474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1330/2642] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:33 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:01:33.890726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:33.901104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c379c3b-502e-401a-b18c-764ca3f1e06c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:33.969632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:33.970308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:33.970595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:33.970872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:33.971568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:33.973543 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:33.995093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1313/2643] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:33 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:34.000331 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:34.011164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4392c48-a8d4-4701-8410-014f16f02cae admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:34.016597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:34.017277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:34.017564 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:34.017839 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:34.018574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:34.020228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:34.031675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1331/2644] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:34.036908 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:34.049753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d8cbb-1a6e-4a64-a9d3-105f904955c2 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:34.054605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad47682-201d-462f-8b96-ed10cd40f726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:34.055293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47682-201d-462f-8b96-ed10cd40f726 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:34.055557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47682-201d-462f-8b96-ed10cd40f726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:34.055832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47682-201d-462f-8b96-ed10cd40f726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:34.056592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad47682-201d-462f-8b96-ed10cd40f726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:34.058202 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad47682-201d-462f-8b96-ed10cd40f726 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:34.070715 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1314/2645] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:34.074917 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:34.085963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba07ee-9ca8-49d8-9e3b-c3ea70a27cc7 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:34.090846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:34.092287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:34.092597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:34.092874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] PATH_INFO: `/v3/projects/5ac05d89cf1a487e9519883c65e9b1f8/users/3c8ccfd159624200a92112f7a43b06f0/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:34.110337 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac05d89cf1a487e9519883c65e9b1f8, user_id=3c8ccfd159624200a92112f7a43b06f0, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:34.113663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:34.131873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1332/2646] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:34 2022] PUT /identity/v3/projects/5ac05d89cf1a487e9519883c65e9b1f8/users/3c8ccfd159624200a92112f7a43b06f0/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:34.145666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c4c1b-ec69-4079-a8a9-b6694967a196 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:34.145962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c4c1b-ec69-4079-a8a9-b6694967a196 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:34.146242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8c4c1b-ec69-4079-a8a9-b6694967a196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:34.173898 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3d8c4c1b-ec69-4079-a8a9-b6694967a196 None None] MFA Rules not processed for user `3c8ccfd159624200a92112f7a43b06f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:34.175175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3d8c4c1b-ec69-4079-a8a9-b6694967a196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:34.243046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1315/2647] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:34.254258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:34.255957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:34.321599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:34.335314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b21dff-f312-4354-a82e-071e56e10eb3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:34.339385 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:34.340104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:34.340389 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:34.340664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:34.350922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:34.352167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:34.362353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1333/2648] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:01:37.628181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:37.628679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:37.628884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:37.648117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] MFA Rules not processed for user `0699a27504da472e96b2dbd52d6cf301`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:37.649260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:37.662089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:37.710517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1316/2649] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:38.035979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:38.037492 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:38.099999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e593f0-a54b-445f-b695-60daf3788061 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:38.104270 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:38.104892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:38.105199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:38.105473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:38.107969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:38.118687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:38.120563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:38.130230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1334/2650] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:38.436624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:38.509124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d00070b-472b-4bc4-b711-c6d92aeb9003 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:38.514858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:38.515546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:38.515829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:38.516132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] PATH_INFO: `/v3/users/3c8ccfd159624200a92112f7a43b06f0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:38.517580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c8ccfd159624200a92112f7a43b06f0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:38.519539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:38.581394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c8ccfd159624200a92112f7a43b06f0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:38.593604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c8ccfd159624200a92112f7a43b06f0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:38.594341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c8ccfd159624200a92112f7a43b06f0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:38.595052 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c8ccfd159624200a92112f7a43b06f0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:38.604428 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-bf93e86f-6b3c-41fa-82c9-a7db20651f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c8ccfd159624200a92112f7a43b06f0'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:38.625447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1317/2651] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:01:38 2022] DELETE /identity/v3/users/3c8ccfd159624200a92112f7a43b06f0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:38.628403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:38.629731 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:38.630184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:38.630334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:38.631199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:38.653717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:38.655341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:38.705930 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:38.716238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216c3e8a-be0d-4bbf-b284-7c9091449d00 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:38.728795 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1318/2652] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:38.738365 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:38.739743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:38.788051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:38.789061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:38.789353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:38.789634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:38.791671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:38.798753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03c529-1bdb-497a-a7da-68205f2a38ae None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:38.802620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e537f-5a45-4ac6-bd56-40854750bd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:38.803268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e537f-5a45-4ac6-bd56-40854750bd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:38.803549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e537f-5a45-4ac6-bd56-40854750bd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:38.803759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:38.803857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e537f-5a45-4ac6-bd56-40854750bd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:38.805409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:38.814053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e537f-5a45-4ac6-bd56-40854750bd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:38.815554 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e537f-5a45-4ac6-bd56-40854750bd1d service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:38.818290 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1335/2653] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:38.825593 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1319/2654] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:01:39.131975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.134526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:39.172767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77690e9a-36c8-461c-8dc9-a72eea311ea6 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.177957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.178832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.179116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.179393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] PATH_INFO: `/v3/projects/5ac05d89cf1a487e9519883c65e9b1f8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.180743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac05d89cf1a487e9519883c65e9b1f8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.185776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.204681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:39.243429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac05d89cf1a487e9519883c65e9b1f8', 'request_id': 'req-e8d3732a-35bc-4a87-8c6d-46117618e6b1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:39.251782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac05d89cf1a487e9519883c65e9b1f8', 'request_id': 'req-e8d3732a-35bc-4a87-8c6d-46117618e6b1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:39.259217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac05d89cf1a487e9519883c65e9b1f8', 'request_id': 'req-e8d3732a-35bc-4a87-8c6d-46117618e6b1'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:39.272986 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] The token cache is being invalidate because project 5ac05d89cf1a487e9519883c65e9b1f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:01:39.273275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:39.273952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-e8d3732a-35bc-4a87-8c6d-46117618e6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:01:39.275511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1336/2655] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:01:39 2022] DELETE /identity/v3/projects/5ac05d89cf1a487e9519883c65e9b1f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:39.281567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.281855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.282387 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.300936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:39.302215 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:39.365188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1320/2656] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:39.373578 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.373870 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.374147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.393831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:39.395053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:39.403499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1337/2657] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:39.407827 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.420130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf5a2b4-b1e3-46fa-a82f-9dad05e664b4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.424917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.425578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.425891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.426167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.427283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:01:39.427595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.429336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.442935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1321/2658] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 20:01:39.447588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.458219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0e5f2-506e-496b-81c8-4bbb7b5f88db None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.462938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.463604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.463892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.464226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.464934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.467328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.493983 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1338/2659] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:01:39 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:01:39.497369 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.509241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94b04-ac57-46d0-9fc5-5254602e1cde admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.570149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.570743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.571031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.571310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.572002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.573962 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.599425 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1322/2660] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:01:39 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:01:39.600673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.612041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04def8-3d5e-4794-9239-50b46ced351c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.617019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.617779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.618074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.618376 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.619149 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.620790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.632574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1339/2661] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:39.636892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.642935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:39.649167 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee52514-f518-49e5-acd0-39d0fca803fe admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.654644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5208c25-985f-4cf7-9124-3931e1dab68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.655325 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5208c25-985f-4cf7-9124-3931e1dab68c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.655612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5208c25-985f-4cf7-9124-3931e1dab68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.655921 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5208c25-985f-4cf7-9124-3931e1dab68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.656696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5208c25-985f-4cf7-9124-3931e1dab68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.658292 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5208c25-985f-4cf7-9124-3931e1dab68c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.672242 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1323/2662] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:01:39.674569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.684884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a741b3e-5187-45b2-ab60-5a6c863926d1 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.690247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.691456 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.691747 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.692028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] PATH_INFO: `/v3/projects/3f5cff0aa09148d2a5e18e557623d0ce/users/90c9fead4dbc418eaaad214d12e91c52/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.707570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f5cff0aa09148d2a5e18e557623d0ce, user_id=90c9fead4dbc418eaaad214d12e91c52, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.710889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.726349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1340/2663] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:01:39 2022] PUT /identity/v3/projects/3f5cff0aa09148d2a5e18e557623d0ce/users/90c9fead4dbc418eaaad214d12e91c52/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:01:39.731565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a6995-9e22-48dc-8ffb-933d0d70812f None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.731888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a6995-9e22-48dc-8ffb-933d0d70812f None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.732742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a6995-9e22-48dc-8ffb-933d0d70812f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.753318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ea3a6995-9e22-48dc-8ffb-933d0d70812f None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:39.754525 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ea3a6995-9e22-48dc-8ffb-933d0d70812f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:39.829357 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1324/2664] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:01:39.839392 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:39.841696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:39.923352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8402d-6fe9-45da-b2e5-6937dcbbe2c9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:39.927507 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:39.928326 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:39.928773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:39.929272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:39.941021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:39.942503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:39.955123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1341/2665] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:01:40.960465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6baab6-a7d7-406f-a7d4-479d32ecb05b None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:40.961950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6baab6-a7d7-406f-a7d4-479d32ecb05b None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:40.962669 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6baab6-a7d7-406f-a7d4-479d32ecb05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:41.010484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8b6baab6-a7d7-406f-a7d4-479d32ecb05b None None] MFA Rules not processed for user `90c9fead4dbc418eaaad214d12e91c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:41.017758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8b6baab6-a7d7-406f-a7d4-479d32ecb05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:41.152225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1325/2666] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:41.165529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:41.167170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:41.232146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:41.251325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe640ffb-6e34-4bc0-be77-f893f4cbbb93 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:41.256287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:41.257145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:41.257634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:41.258037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:41.270822 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:41.271754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:41.282423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1342/2667] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:43.717524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:43.718020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:43.718117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:43.742110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] MFA Rules not processed for user `90c9fead4dbc418eaaad214d12e91c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:01:43.743608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:43.752562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1326/2668] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:01:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:01:43.985211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:43.986698 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:43.998892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaebe32-97b6-4f36-8691-0d67324f972e service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:44.003746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:44.004485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:44.004764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:44.005038 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:44.007073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:44.020221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:44.021727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:44.030359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1343/2669] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:52.774915 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:52.777237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:52.882743 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae80ee9-4517-4282-bcaa-1567ede6602d None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:52.890772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:52.892630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:52.892938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:52.893246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:52.895939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:52.913635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:52.916473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:52.924127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=100 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:52.931299 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1327/2670] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:01:54.026467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=13 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:01:57.942905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:01:57.951661 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fc1a2-2f17-400f-ae4b-06ae85078d08 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:01:57.956163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:01:57.956814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:01:57.957094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:01:57.957369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:01:57.959145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:01:58.027563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:01:58.029197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c68766-6f04-48c1-811c-dae2d6de5e7c service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:01:58.040451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1344/2671] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:01:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:02:00.242314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:00.244411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:00.261044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbb38de-418d-45c4-8ede-5e7ce223f5fd service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:00.267875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:00.268782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:00.269097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:00.269380 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:00.271398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:00.287178 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:00.288998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:00.303151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1328/2672] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:02:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:02:06.017452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058000cc-09cb-4099-b40f-d6f58911dbb4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:06.017955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058000cc-09cb-4099-b40f-d6f58911dbb4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:06.018117 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058000cc-09cb-4099-b40f-d6f58911dbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:06.036845 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-058000cc-09cb-4099-b40f-d6f58911dbb4 None None] MFA Rules not processed for user `90c9fead4dbc418eaaad214d12e91c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:02:06.038313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-058000cc-09cb-4099-b40f-d6f58911dbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:06.045450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1345/2673] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:02:10.299455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:02:15.887549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:15.897126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1e4a8-d7cd-41e8-94f3-b57dab7886be service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:15.900816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:15.901456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:15.901738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:15.902088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:15.903882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:15.970442 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:15.971856 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:15.981575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1329/2674] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:02:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:02:16.030689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=33 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:02:25.977103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=30 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:02:28.621554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:28.622024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:28.622131 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:28.682808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:02:28.684008 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:28.751366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1346/2675] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 20:02:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:02:28.798345 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:28.799890 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:28.896256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac24a3e-6c0a-47ac-b21a-9dbda950e876 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:28.901880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b08f0-224d-4a41-bf87-2e5b35ad1242 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:28.902689 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b08f0-224d-4a41-bf87-2e5b35ad1242 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:28.903323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b08f0-224d-4a41-bf87-2e5b35ad1242 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:28.903589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b08f0-224d-4a41-bf87-2e5b35ad1242 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:28.916819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b08f0-224d-4a41-bf87-2e5b35ad1242 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:28.918439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b08f0-224d-4a41-bf87-2e5b35ad1242 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:28.933396 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1330/2676] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 20:02:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 20:02:35.451523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:35.456154 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:35.473346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e944d-1be4-4b99-8ad8-d0efbb41e5c1 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:35.476940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:35.477602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:35.477886 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:35.478163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:35.479961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:35.618415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:35.621022 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:35.634499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1347/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:02:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:02:38.929803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=41 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:02:43.929575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:43.930189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:43.930189 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:43.948873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:02:43.950074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:43.957176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1331/2678] 10.222.0.17 () {58 vars in 967 bytes} [Fri Jul 15 20:02:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:02:43.969630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:43.981183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a6c76-4a21-458a-9318-9d38a1cac2df service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:43.986849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:43.987487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:43.987795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:43.988098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:43.996681 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:43.998284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:44.006400 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1348/2679] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 20:02:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 20:02:45.851680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:45.853289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:45.928311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:45.941226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266da7d3-403e-40fb-ba86-c185842febc9 service swift] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:45.945828 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:45.946764 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:45.947050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:45.947323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] PATH_INFO: `/v3/users/90c9fead4dbc418eaaad214d12e91c52` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:45.948584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c9fead4dbc418eaaad214d12e91c52)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:45.950552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:45.952342 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f970d0a-ba2a-4320-a09b-2708959d4be4 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:45.957460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:45.958236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:45.958518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:45.958794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:45.960897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:02:45.973464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:45.975377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:45.988480 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1332/2680] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:02:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:02:45.991885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:02:46.006997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c9fead4dbc418eaaad214d12e91c52'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.015043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c9fead4dbc418eaaad214d12e91c52'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.015804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c9fead4dbc418eaaad214d12e91c52'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.017072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c9fead4dbc418eaaad214d12e91c52'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.031603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c9fead4dbc418eaaad214d12e91c52'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.043783 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1349/2681] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:02:45 2022] DELETE /identity/v3/users/90c9fead4dbc418eaaad214d12e91c52 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:02:46.220483 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:02:46.231257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021aefd-b03d-4703-bf97-20dee444e68d service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:02:46.294794 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:02:46.296281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:02:46.296742 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:02:46.297069 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] PATH_INFO: `/v3/projects/3f5cff0aa09148d2a5e18e557623d0ce` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:02:46.298309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f5cff0aa09148d2a5e18e557623d0ce)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:02:46.300270 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:02:46.347153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f5cff0aa09148d2a5e18e557623d0ce', 'request_id': 'req-1c080000-2cb6-4e97-bb9b-59f483335251'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.353070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f5cff0aa09148d2a5e18e557623d0ce', 'request_id': 'req-1c080000-2cb6-4e97-bb9b-59f483335251'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.359479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f5cff0aa09148d2a5e18e557623d0ce', 'request_id': 'req-1c080000-2cb6-4e97-bb9b-59f483335251'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.372562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] The token cache is being invalidate because project 3f5cff0aa09148d2a5e18e557623d0ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:02:46.372851 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.373681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:02:46.375324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1333/2682] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:02:46 2022] DELETE /identity/v3/projects/3f5cff0aa09148d2a5e18e557623d0ce => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:02:56.040142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=17,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:02:56.369184 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:05.270179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:05.271636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:05.328264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc38761-877e-4d00-98a3-6070285c9950 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:05.376799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:05.377450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:05.377738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:05.378018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:05.379819 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:05.443171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:05.444640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:05.454852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1350/2683] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:08.625373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:08.627029 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:08.693432 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:08.695379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:08.706294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c080000-2cb6-4e97-bb9b-59f483335251 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:08.708513 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a86cd3-94ea-4c88-87de-d4b9157f2dca service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:08.712062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:08.712929 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:08.713263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:08.713361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:08.713643 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:08.714367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:08.714718 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:08.715064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] PATH_INFO: `/v3/users/bf32cd62eec04de892c3b27fbdd882a8` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:08.715779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:08.716484 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf32cd62eec04de892c3b27fbdd882a8)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:08.718692 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:08.737208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:08.739487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:08.756725 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1334/2684] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:04:08.778928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf32cd62eec04de892c3b27fbdd882a8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:08.787748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf32cd62eec04de892c3b27fbdd882a8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:08.789077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf32cd62eec04de892c3b27fbdd882a8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:08.789757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf32cd62eec04de892c3b27fbdd882a8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:08.795314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:08.804379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-c2c3dd03-f503-46d3-b241-98503a8c1d60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf32cd62eec04de892c3b27fbdd882a8'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:08.819478 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1351/2685] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:08 2022] DELETE /identity/v3/users/bf32cd62eec04de892c3b27fbdd882a8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:09.003512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.019051 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ec55c-ee79-4573-8af0-01031243622c service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.082634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.083497 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.083785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.084065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] PATH_INFO: `/v3/projects/ebcd8788770344d290ebf376ec33e7ad` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.085281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebcd8788770344d290ebf376ec33e7ad)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.087208 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.135612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebcd8788770344d290ebf376ec33e7ad', 'request_id': 'req-c8803576-74b2-46e8-b302-83ae516811bc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:09.141652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebcd8788770344d290ebf376ec33e7ad', 'request_id': 'req-c8803576-74b2-46e8-b302-83ae516811bc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:09.147229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebcd8788770344d290ebf376ec33e7ad', 'request_id': 'req-c8803576-74b2-46e8-b302-83ae516811bc'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:09.160393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] The token cache is being invalidate because project ebcd8788770344d290ebf376ec33e7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:09.160683 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:09.161316 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c8803576-74b2-46e8-b302-83ae516811bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:09.162877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1335/2686] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:09 2022] DELETE /identity/v3/projects/ebcd8788770344d290ebf376ec33e7ad => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:09.168855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.169166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.169467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.188663 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:09.189940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:09.251367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1352/2687] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:09.258839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.259132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.259413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.278839 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:09.280848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:09.289045 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1336/2688] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:09.293340 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.304065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e71d47-1593-4d59-bf52-6b462a4a6be6 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.310967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.311740 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.312100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.312403 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.313268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:04:09.313609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.315616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.330493 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1353/2689] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:04:09.334965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.346294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01872ee-602c-4149-806c-1bc5330f3b11 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.351688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.352508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.352863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.353206 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.354054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.356468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.368278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:09.386597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1337/2690] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:09 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:09.391775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.401789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca783ce-e3e1-4053-995d-6477f4fe93a3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.460708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.461380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.461664 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.461942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.462633 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.464779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.484486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1354/2691] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:09 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:04:09.488968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.500123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e33c0c-6a39-4dab-8301-1ed4956a9e60 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.504750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.505412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.505696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.505992 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.506705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.508352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.519775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1338/2692] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:09.523919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.534521 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15e8a5-fde8-44ea-8345-f09140c0ade9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.539428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba941f37-6276-4701-a44c-bc69c25eda7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.540100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941f37-6276-4701-a44c-bc69c25eda7a admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.540408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941f37-6276-4701-a44c-bc69c25eda7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.540688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba941f37-6276-4701-a44c-bc69c25eda7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.541450 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941f37-6276-4701-a44c-bc69c25eda7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.543178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba941f37-6276-4701-a44c-bc69c25eda7a admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.554833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1355/2693] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:09.558582 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.569920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91407d6e-3aed-41e3-94f1-3ec1e1a892ae admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.574718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.575597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.575881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.576187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] PATH_INFO: `/v3/projects/3119f756466b4856bc4b546a7d91e0ef/users/00ea93bc738840cba9117c29d2ece603/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.593306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3119f756466b4856bc4b546a7d91e0ef, user_id=00ea93bc738840cba9117c29d2ece603, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.596821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.613691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1339/2694] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:09 2022] PUT /identity/v3/projects/3119f756466b4856bc4b546a7d91e0ef/users/00ea93bc738840cba9117c29d2ece603/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:04:09.618735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd314f-3897-4846-83f4-cf4b67d123bd None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.619067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd314f-3897-4846-83f4-cf4b67d123bd None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.619359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd314f-3897-4846-83f4-cf4b67d123bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.637724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-68fd314f-3897-4846-83f4-cf4b67d123bd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:09.638856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-68fd314f-3897-4846-83f4-cf4b67d123bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:09.681472 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:09.703599 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1356/2695] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:04:09.714465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:09.716303 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:09.783455 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc10e0-3242-4ad4-b09e-a86a3aa2a512 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:09.787341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:09.787978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:09.788233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:09.788513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:09.797843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:09.799315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:09.808971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1340/2696] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:10.432695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.432965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.433243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.461421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] MFA Rules not processed for user `00ea93bc738840cba9117c29d2ece603`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:10.462635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:10.532042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1357/2697] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:10 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:04:10.536624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:10.547627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01498576-4c7c-434d-9c7f-b9c2f5577bb3 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:10.552454 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:10.553127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.553459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.553766 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.554501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:10.556660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:10.587925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1341/2698] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:10 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:10.589235 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:10.600855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6497d3-030e-4545-9621-2280abb37f54 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:10.662134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:10.662802 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.663085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.663362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.664048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:10.666037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:10.687580 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1358/2699] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:10 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:10.691853 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:10.703126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c5263-6feb-4d8e-9f82-45ccbc4d85bf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:10.708061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:10.708793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.709086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.709377 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.710170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:10.711882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:10.725088 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1342/2700] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:10.729252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:10.739469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb390f-30c2-4a3b-8bfa-b26b55b6d16d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:10.744242 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:10.745399 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.745679 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.745951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] PATH_INFO: `/v3/projects/96c0a67251b249f7849d27696c2709ac/users/cdaf5af2744d44cc82ab8c04f1b6122f/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.761086 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c0a67251b249f7849d27696c2709ac, user_id=cdaf5af2744d44cc82ab8c04f1b6122f, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:10.763823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:10.781951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1359/2701] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:10 2022] PUT /identity/v3/projects/96c0a67251b249f7849d27696c2709ac/users/cdaf5af2744d44cc82ab8c04f1b6122f/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:10.786268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:10.797075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51a17f-f8c0-48ed-a581-209543c22b9e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:10.856462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322447e-ca6c-40cf-9fe1-bb4fd3c41909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:10.857131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322447e-ca6c-40cf-9fe1-bb4fd3c41909 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.857412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322447e-ca6c-40cf-9fe1-bb4fd3c41909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.857732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322447e-ca6c-40cf-9fe1-bb4fd3c41909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.859124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322447e-ca6c-40cf-9fe1-bb4fd3c41909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:10.860754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322447e-ca6c-40cf-9fe1-bb4fd3c41909 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:10.870834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1343/2702] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:10.875144 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:10.885349 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3fd68-5194-4664-bef6-8810b6e758f9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:10.890632 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:10.891510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:10.891801 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:10.892062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] PATH_INFO: `/v3/projects/96c0a67251b249f7849d27696c2709ac/users/cdaf5af2744d44cc82ab8c04f1b6122f/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:10.894046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c0a67251b249f7849d27696c2709ac, user_id=cdaf5af2744d44cc82ab8c04f1b6122f, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:10.896814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:10.907628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1360/2703] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:10 2022] PUT /identity/v3/projects/96c0a67251b249f7849d27696c2709ac/users/cdaf5af2744d44cc82ab8c04f1b6122f/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:04:11.513096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:11.513415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:11.513699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:11.541472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] MFA Rules not processed for user `cdaf5af2744d44cc82ab8c04f1b6122f`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:11.542685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:11.552042 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:11.620456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1344/2704] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:12.696969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:12.733369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:12.768627 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651f830b-d56f-4860-af28-260152a2d162 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:12.773188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:12.774063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:12.774344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:12.774620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] PATH_INFO: `/v3/users/00ea93bc738840cba9117c29d2ece603` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:12.775856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00ea93bc738840cba9117c29d2ece603)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:12.777852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:12.825883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00ea93bc738840cba9117c29d2ece603'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:12.834313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00ea93bc738840cba9117c29d2ece603'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:12.835028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00ea93bc738840cba9117c29d2ece603'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:12.835705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00ea93bc738840cba9117c29d2ece603'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:12.849474 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00ea93bc738840cba9117c29d2ece603'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:12.861451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1361/2705] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:12 2022] DELETE /identity/v3/users/00ea93bc738840cba9117c29d2ece603 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:13.041430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:13.052665 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704317d3-f791-433c-a755-6caa54bfad6c None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:13.114235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:13.115108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:13.115395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:13.115667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] PATH_INFO: `/v3/projects/3119f756466b4856bc4b546a7d91e0ef` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:13.116896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3119f756466b4856bc4b546a7d91e0ef)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:13.118854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:13.170922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3119f756466b4856bc4b546a7d91e0ef', 'request_id': 'req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.177065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3119f756466b4856bc4b546a7d91e0ef', 'request_id': 'req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.182564 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3119f756466b4856bc4b546a7d91e0ef', 'request_id': 'req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.195503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] The token cache is being invalidate because project 3119f756466b4856bc4b546a7d91e0ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:13.195816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.196467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.197997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1345/2706] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:13 2022] DELETE /identity/v3/projects/3119f756466b4856bc4b546a7d91e0ef => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:13.202894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:13.204315 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:13.218166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f3ff1e-b5f9-4ff1-ae15-8fe14e99bbc0 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:13.277091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:13.278496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:13.279100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:13.279567 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] PATH_INFO: `/v3/users/cdaf5af2744d44cc82ab8c04f1b6122f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:13.281709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdaf5af2744d44cc82ab8c04f1b6122f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:13.284540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:13.340537 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdaf5af2744d44cc82ab8c04f1b6122f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.342265 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:13.349292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdaf5af2744d44cc82ab8c04f1b6122f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.350946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdaf5af2744d44cc82ab8c04f1b6122f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.351995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdaf5af2744d44cc82ab8c04f1b6122f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.370383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-0132637c-95a1-4b60-af6d-f50446d22ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdaf5af2744d44cc82ab8c04f1b6122f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.386619 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1362/2707] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:13 2022] DELETE /identity/v3/users/cdaf5af2744d44cc82ab8c04f1b6122f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:13.622720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:13.625363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:13.644335 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3dfb6-bcde-45a5-9150-a40b4a4dac4f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:13.689059 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:13.725280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:13.726295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:13.726549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:13.726841 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] PATH_INFO: `/v3/projects/96c0a67251b249f7849d27696c2709ac` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:13.728254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c0a67251b249f7849d27696c2709ac)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:13.730475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:13.795040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c0a67251b249f7849d27696c2709ac', 'request_id': 'req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.805099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c0a67251b249f7849d27696c2709ac', 'request_id': 'req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.813422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c0a67251b249f7849d27696c2709ac', 'request_id': 'req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.828359 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] The token cache is being invalidate because project 96c0a67251b249f7849d27696c2709ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:13.828645 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.829263 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-5c3e60af-91da-4120-b61c-bfa62ec8cf9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:13.830861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1346/2708] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:13 2022] DELETE /identity/v3/projects/96c0a67251b249f7849d27696c2709ac => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:13.837136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:13.837675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:13.837675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:13.855829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:13.857261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:13.925118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1363/2709] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:13.933086 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:13.933408 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:13.933688 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:13.954214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:13.955456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:13.963535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1347/2710] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:13.968382 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:13.978570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245e94ee-b961-44a6-8fa2-5e19902805ca None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:13.983768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:13.984566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:13.984852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:13.985127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:13.985868 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:04:13.986174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:13.987865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.001850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1364/2711] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:04:14.006363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.017464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c73aab-fd73-41c7-9a26-8fb8f03248b5 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.022108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.022779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.023087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.023363 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.024070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.026019 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.053406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1348/2712] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:14 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:14.058015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.069761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d28fc78-7615-416c-ba16-daa7d2ab6d3c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.131715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.132422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.132707 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.132985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.133770 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.136231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.158572 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1365/2713] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:14 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:14.162653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.173220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145a1e2f-ee35-4685-9d32-e272fabf4d5d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.177740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.178429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.178711 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.178987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.179710 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.181353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.193154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1349/2714] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:14.197197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.207810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6638a-76d6-4b13-9ae7-13b56a78f98d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.212404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67e61f-0fb2-41f3-a2ad-3729c2d6b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.213063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67e61f-0fb2-41f3-a2ad-3729c2d6b66e admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.213352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67e61f-0fb2-41f3-a2ad-3729c2d6b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.213630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67e61f-0fb2-41f3-a2ad-3729c2d6b66e admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.214358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67e61f-0fb2-41f3-a2ad-3729c2d6b66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.215969 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67e61f-0fb2-41f3-a2ad-3729c2d6b66e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.226028 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1366/2715] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:04:14.230998 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.241133 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746eef02-c1cc-419e-8861-f368a3a3cee1 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.245985 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.246867 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.247172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.247459 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] PATH_INFO: `/v3/projects/8ab06f86281a4773ba02e59531fcd148/users/0ab8a2f5c12b48da9936be22b5fb8867/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.264797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab06f86281a4773ba02e59531fcd148, user_id=0ab8a2f5c12b48da9936be22b5fb8867, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.267775 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.282073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1350/2716] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:14 2022] PUT /identity/v3/projects/8ab06f86281a4773ba02e59531fcd148/users/0ab8a2f5c12b48da9936be22b5fb8867/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:14.294042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.294336 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.294622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.323735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] MFA Rules not processed for user `0ab8a2f5c12b48da9936be22b5fb8867`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:14.325347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:14.346508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:14.395514 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1367/2717] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:14.399952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.462605 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc97473-c8b9-4af1-bd51-4a2b8fb7722b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.468210 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.469108 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.469398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.469675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.470361 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.472306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.480430 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:14.500249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1351/2718] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:14 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:14.504283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.516026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51aed1-2960-491e-9507-e8bd8ae4f107 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.574696 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.575381 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.575666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.575944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.577209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.579221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.599915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1368/2719] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:14 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:14.603886 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.615125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d44e58-8f63-4126-97a6-cb3e94dc158e admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.619680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.620639 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.620920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.621196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.621951 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.624304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.635963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1352/2720] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:14.640217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.654942 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eef938-a374-4641-b891-f65cff282cef admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.661164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.662119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.662413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.662675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] PATH_INFO: `/v3/projects/f4b7afa3337f412f8ca298801134e86a/users/59a1c164dcf74807bfd86f104a14df24/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.681051 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b7afa3337f412f8ca298801134e86a, user_id=59a1c164dcf74807bfd86f104a14df24, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.685186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.702072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1369/2721] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:14 2022] PUT /identity/v3/projects/f4b7afa3337f412f8ca298801134e86a/users/59a1c164dcf74807bfd86f104a14df24/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:14.706028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.716774 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799f44e-4f95-4066-99be-1acc277c7cfc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.775786 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3818c274-3b44-44a6-8903-2900e873aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.776493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c274-3b44-44a6-8903-2900e873aaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.776778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c274-3b44-44a6-8903-2900e873aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.777054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3818c274-3b44-44a6-8903-2900e873aaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.777777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818c274-3b44-44a6-8903-2900e873aaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.779501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3818c274-3b44-44a6-8903-2900e873aaaf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.790519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1353/2722] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:14.794726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:14.806150 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4775ab-72e7-4912-adf9-74f21e92ee9f admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:14.811351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:14.812358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.812640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.812913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] PATH_INFO: `/v3/projects/f4b7afa3337f412f8ca298801134e86a/users/59a1c164dcf74807bfd86f104a14df24/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.817850 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b7afa3337f412f8ca298801134e86a, user_id=59a1c164dcf74807bfd86f104a14df24, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:14.820736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:14.831240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1370/2723] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:14 2022] PUT /identity/v3/projects/f4b7afa3337f412f8ca298801134e86a/users/59a1c164dcf74807bfd86f104a14df24/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:14.891127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:14.891809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:14.891809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:14.918797 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] MFA Rules not processed for user `59a1c164dcf74807bfd86f104a14df24`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:14.920020 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:14.990117 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1354/2724] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:15.002496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:15.004306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:15.061297 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbb5f9-82cc-4c64-82d1-03b26e472bde admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:15.065383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634c29ed-4f93-4a2c-b9ae-d8d6254d6de3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:15.066058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c29ed-4f93-4a2c-b9ae-d8d6254d6de3 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:15.066343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c29ed-4f93-4a2c-b9ae-d8d6254d6de3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:15.066622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634c29ed-4f93-4a2c-b9ae-d8d6254d6de3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:15.075820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c29ed-4f93-4a2c-b9ae-d8d6254d6de3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:15.077907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634c29ed-4f93-4a2c-b9ae-d8d6254d6de3 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:15.087320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1371/2725] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:15 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:15.114935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:15.176670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b70f78-8634-4e90-9191-aa8e5c51dc35 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:15.181236 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:15.182254 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:15.182538 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:15.182806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] PATH_INFO: `/v3/users/0ab8a2f5c12b48da9936be22b5fb8867` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:15.184001 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab8a2f5c12b48da9936be22b5fb8867)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:15.186626 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:15.188324 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:15.231835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab8a2f5c12b48da9936be22b5fb8867'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.240597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab8a2f5c12b48da9936be22b5fb8867'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.241229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab8a2f5c12b48da9936be22b5fb8867'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.241858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab8a2f5c12b48da9936be22b5fb8867'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.250258 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab8a2f5c12b48da9936be22b5fb8867'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.268137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1355/2726] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:15 2022] DELETE /identity/v3/users/0ab8a2f5c12b48da9936be22b5fb8867 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:15.272658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:15.272951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:15.273234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:15.291063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:15.292355 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:15.302463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:15.352812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1372/2727] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:15.363315 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:15.364754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:15.423287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf6ce-e9f5-4fe7-afad-4666d2ea6c50 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:15.426909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:15.427588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:15.427874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:15.428187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:15.437806 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:15.439240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:15.449027 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1356/2728] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:15.744540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:15.746240 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:15.760541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3fb80-8e11-4f81-ac6e-ba60a2b6c9c3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:15.765643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:15.766579 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:15.766815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:15.767097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] PATH_INFO: `/v3/projects/8ab06f86281a4773ba02e59531fcd148` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:15.768343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab06f86281a4773ba02e59531fcd148)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:15.775151 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:15.821984 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab06f86281a4773ba02e59531fcd148', 'request_id': 'req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.829683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab06f86281a4773ba02e59531fcd148', 'request_id': 'req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.835851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab06f86281a4773ba02e59531fcd148', 'request_id': 'req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.846232 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] The token cache is being invalidate because project 8ab06f86281a4773ba02e59531fcd148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:15.846522 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.847169 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.848752 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1373/2729] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:15 2022] DELETE /identity/v3/projects/8ab06f86281a4773ba02e59531fcd148 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:15.852855 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:15.862835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414764e-d09e-4695-a282-2abe598dc577 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:15.926050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:15.926927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:15.927214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:15.927493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] PATH_INFO: `/v3/users/59a1c164dcf74807bfd86f104a14df24` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:15.928724 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59a1c164dcf74807bfd86f104a14df24)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:15.930802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:15.954462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:15.980057 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a1c164dcf74807bfd86f104a14df24'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.987934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a1c164dcf74807bfd86f104a14df24'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.988578 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a1c164dcf74807bfd86f104a14df24'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.989274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a1c164dcf74807bfd86f104a14df24'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:15.996684 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9c2d1769-8047-4a5f-b0fa-3aff0e9f7b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a1c164dcf74807bfd86f104a14df24'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:16.014328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1357/2730] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:15 2022] DELETE /identity/v3/users/59a1c164dcf74807bfd86f104a14df24 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:04:16.294673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.296469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:16.314762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b007da7-f7d4-4d27-acd5-e0ef22c0e501 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.377063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.378052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.378328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.378605 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] PATH_INFO: `/v3/projects/f4b7afa3337f412f8ca298801134e86a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.379773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b7afa3337f412f8ca298801134e86a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.381731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.423095 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:16.429897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b7afa3337f412f8ca298801134e86a', 'request_id': 'req-d2087675-ab7d-4bdf-a63e-91e4783661d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:16.437796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b7afa3337f412f8ca298801134e86a', 'request_id': 'req-d2087675-ab7d-4bdf-a63e-91e4783661d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:16.444414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b7afa3337f412f8ca298801134e86a', 'request_id': 'req-d2087675-ab7d-4bdf-a63e-91e4783661d5'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:16.455467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] The token cache is being invalidate because project f4b7afa3337f412f8ca298801134e86a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:16.455750 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:16.456388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-d2087675-ab7d-4bdf-a63e-91e4783661d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:16.457902 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1374/2731] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:16 2022] DELETE /identity/v3/projects/f4b7afa3337f412f8ca298801134e86a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:16.462938 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.463386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.463489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.481656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:16.483066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:16.543320 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1358/2732] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:16.551716 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.552003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.552308 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.570813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:16.572007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:16.580710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1375/2733] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:16.584895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.594548 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555844e-498a-4053-a5cb-6202bbb6c2d6 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.599032 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.599677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.599958 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.600276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.601002 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:04:16.601312 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.602937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.614658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1359/2734] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:04:16.618946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.629479 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c41bbd-5acc-4a0c-80b5-8f51b9580cde None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.633970 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.634617 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.634896 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.635285 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.635967 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.637906 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.665412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1376/2735] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:16 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:16.669281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.679697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cddbfb-1259-4b3c-be4c-d5d44bdf5863 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.737024 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.737694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.737977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.738252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.738947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.740919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.761216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1360/2736] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:16 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:04:16.765864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.776480 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd6dc5-36b5-4559-a1cc-d5b63baee421 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.782499 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.783314 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.783636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.783961 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.784843 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.786670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.797682 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1377/2737] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:16.801880 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.812875 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70178a-c005-4f6f-aec5-0c5b027202aa admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.817991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7412f-3222-4407-b8a2-607f81ac330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.818772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7412f-3222-4407-b8a2-607f81ac330d admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.819096 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7412f-3222-4407-b8a2-607f81ac330d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.819439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7412f-3222-4407-b8a2-607f81ac330d admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.820309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7412f-3222-4407-b8a2-607f81ac330d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.822191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd7412f-3222-4407-b8a2-607f81ac330d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.835382 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1361/2738] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:04:16.839566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:16.850416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb8a78-7978-4b82-8d4a-accb94aa9748 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:16.855062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:16.855939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.856258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.856538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] PATH_INFO: `/v3/projects/cb8afc64593e49d8aef0479feb509848/users/02262516b3284eaf8f81090f0aac2be5/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.871595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8afc64593e49d8aef0479feb509848, user_id=02262516b3284eaf8f81090f0aac2be5, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:16.875032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:16.887955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1378/2739] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:16 2022] PUT /identity/v3/projects/cb8afc64593e49d8aef0479feb509848/users/02262516b3284eaf8f81090f0aac2be5/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:16.901732 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:16.902026 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:16.902326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:16.915858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:16.931250 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] MFA Rules not processed for user `02262516b3284eaf8f81090f0aac2be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:16.932521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:17.003712 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1362/2740] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:16 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:17.009199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.075840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16859c58-e377-4831-87d5-b6a86566f041 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.081057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.081709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.081992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.082267 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.082946 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.084909 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.110945 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1379/2741] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:17 2022] POST /identity/v3/projects => generated 401 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:17.115132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.126557 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c44e2f-820b-408e-9ac5-162e2ea605f9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.182329 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.182932 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.183220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.183501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.184224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.187134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.206961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1363/2742] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:17 2022] POST /identity/v3/users => generated 307 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:17.210919 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.221057 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f6f58-a16e-408e-b941-81561a2f145c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.225615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.226271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.226553 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.226831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.227554 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.229204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.240471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1380/2743] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:17.244600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.256034 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0eca3-58a4-4663-90fe-48c33b7d6c56 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.260922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.261835 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.262120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.262395 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] PATH_INFO: `/v3/projects/e00a451f424049ac8e2b0490dd15d7f3/users/1f1f14bd81d34f94a60d34705be16e12/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.278402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00a451f424049ac8e2b0490dd15d7f3, user_id=1f1f14bd81d34f94a60d34705be16e12, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.281129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.295328 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1364/2744] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:17 2022] PUT /identity/v3/projects/e00a451f424049ac8e2b0490dd15d7f3/users/1f1f14bd81d34f94a60d34705be16e12/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:17.299486 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.309660 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f487e0-b2ed-4ef4-8782-5a65807f8304 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.351775 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:17.372678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b9456-0aa3-4870-b234-64b1b600c4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.374106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b9456-0aa3-4870-b234-64b1b600c4a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.374106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b9456-0aa3-4870-b234-64b1b600c4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.374379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b9456-0aa3-4870-b234-64b1b600c4a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.375096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b9456-0aa3-4870-b234-64b1b600c4a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.376738 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b9456-0aa3-4870-b234-64b1b600c4a2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.386503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1381/2745] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:17.392239 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.402621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01186f-80c2-4e06-b889-fc8b65f92040 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.407598 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.408502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.408790 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.409066 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] PATH_INFO: `/v3/projects/e00a451f424049ac8e2b0490dd15d7f3/users/1f1f14bd81d34f94a60d34705be16e12/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.411035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00a451f424049ac8e2b0490dd15d7f3, user_id=1f1f14bd81d34f94a60d34705be16e12, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.413760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.424559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1365/2746] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:17 2022] PUT /identity/v3/projects/e00a451f424049ac8e2b0490dd15d7f3/users/1f1f14bd81d34f94a60d34705be16e12/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:17.434556 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.434777 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.435062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.462318 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] MFA Rules not processed for user `1f1f14bd81d34f94a60d34705be16e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:17.463536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:17.534414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1382/2747] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:17 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:17.544889 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.546846 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:17.600421 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:17.604468 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72afe8e-1c59-4802-b2e0-dc9e468b6f3c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.610259 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.610946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.611235 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.611516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.621200 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.622653 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.633604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1366/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:17 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:17.708404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:17.719935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f589f9-146c-4e35-93bd-84e07f412294 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:17.723951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c74864-8b8e-4177-afda-adf8000da293 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:17.724638 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74864-8b8e-4177-afda-adf8000da293 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:17.724955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74864-8b8e-4177-afda-adf8000da293 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:17.725333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74864-8b8e-4177-afda-adf8000da293 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:17.790459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c74864-8b8e-4177-afda-adf8000da293 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:17.791925 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c74864-8b8e-4177-afda-adf8000da293 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:17.801548 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1383/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:22.052770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:22.122634 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ee98-96b9-4f06-a1c4-3574aa05887c service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:22.128652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:22.129514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:22.129796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:22.130072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] PATH_INFO: `/v3/users/02262516b3284eaf8f81090f0aac2be5` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:22.131504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02262516b3284eaf8f81090f0aac2be5)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:22.133858 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:22.184141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02262516b3284eaf8f81090f0aac2be5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.192592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02262516b3284eaf8f81090f0aac2be5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.193297 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02262516b3284eaf8f81090f0aac2be5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.193950 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02262516b3284eaf8f81090f0aac2be5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.202680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02262516b3284eaf8f81090f0aac2be5'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.221214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1367/2750] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:22 2022] DELETE /identity/v3/users/02262516b3284eaf8f81090f0aac2be5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:22.225885 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:22.226354 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:22.226442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:22.246065 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:22.247547 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:22.300526 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:22.309155 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1384/2751] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:22.319544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:22.321033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:22.392838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763588-908b-4fa4-9e61-05c2da9998c8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:22.396512 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:22.397162 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:22.397449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:22.397730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:22.407326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:22.408787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:22.412777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:22.420609 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1368/2752] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:04:22.689327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:22.691010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:22.710072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e166aa6-4840-4ceb-b61d-20fed8105860 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:22.714756 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:22.715813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:22.716220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:22.716630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] PATH_INFO: `/v3/projects/cb8afc64593e49d8aef0479feb509848` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:22.718325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8afc64593e49d8aef0479feb509848)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:22.720683 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:22.773123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8afc64593e49d8aef0479feb509848', 'request_id': 'req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.780768 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8afc64593e49d8aef0479feb509848', 'request_id': 'req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.787391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8afc64593e49d8aef0479feb509848', 'request_id': 'req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.798358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] The token cache is being invalidate because project cb8afc64593e49d8aef0479feb509848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:22.798646 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.799364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.800836 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1385/2753] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:22 2022] DELETE /identity/v3/projects/cb8afc64593e49d8aef0479feb509848 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:22.805043 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:22.815948 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bcf32e-9a7a-4eb4-b86a-85fa114ccc57 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:22.876675 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:22.877600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:22.877891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:22.878170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] PATH_INFO: `/v3/users/1f1f14bd81d34f94a60d34705be16e12` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:22.879366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1f14bd81d34f94a60d34705be16e12)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:22.881337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:22.928176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1f14bd81d34f94a60d34705be16e12'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.938187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1f14bd81d34f94a60d34705be16e12'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.938876 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1f14bd81d34f94a60d34705be16e12'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.939500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1f14bd81d34f94a60d34705be16e12'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.946727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1d44cdf6-638c-4810-9aee-9e9c52fafdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1f14bd81d34f94a60d34705be16e12'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:22.965048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1369/2754] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:22 2022] DELETE /identity/v3/users/1f1f14bd81d34f94a60d34705be16e12 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:23.276642 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.278311 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:23.292410 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca0b8c9-e7a7-497e-8c9f-44b184d87012 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.354120 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.355062 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.355362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.355657 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] PATH_INFO: `/v3/projects/e00a451f424049ac8e2b0490dd15d7f3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.356943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00a451f424049ac8e2b0490dd15d7f3)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.359934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.407429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00a451f424049ac8e2b0490dd15d7f3', 'request_id': 'req-899931dd-6a97-4e39-8fa2-0ffdc6c99070'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:23.413769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00a451f424049ac8e2b0490dd15d7f3', 'request_id': 'req-899931dd-6a97-4e39-8fa2-0ffdc6c99070'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:23.419945 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00a451f424049ac8e2b0490dd15d7f3', 'request_id': 'req-899931dd-6a97-4e39-8fa2-0ffdc6c99070'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:23.431451 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] The token cache is being invalidate because project e00a451f424049ac8e2b0490dd15d7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:23.431737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:23.432394 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-899931dd-6a97-4e39-8fa2-0ffdc6c99070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:23.433928 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1386/2755] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:23 2022] DELETE /identity/v3/projects/e00a451f424049ac8e2b0490dd15d7f3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:23.439674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.440163 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.440280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.458624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:23.459816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:23.521295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1370/2756] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:04:23.529323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.529552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.529867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.548790 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:23.550826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:23.558830 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1387/2757] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:23.563383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.574352 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6a28e-080b-4561-84fd-150351177d69 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.579590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.580283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.580568 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.580837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.581569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:04:23.581872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.583990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.596607 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1371/2758] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 15 20:04:23.600965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.610709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d00b144-a38a-4167-8ca4-5d292d7d93c9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.615402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.616055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.616373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.616652 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.617353 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.619268 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.622831 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:23.652129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1388/2759] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:23 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:23.653623 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.661009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:23.665478 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c53e028-7824-45f9-98f2-7e763691bd21 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.728207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.728887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.729174 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.729457 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.730153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.732118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.755118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1372/2760] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:23 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:04:23.759136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.770162 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82abba-7245-411c-b70a-5aa029af12e4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.774892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.775519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.775799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.776094 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.776838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.778485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.790482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1389/2761] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:23.794422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.804955 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbcb4c-86e8-4e50-9731-a8e9c1acccc5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.809606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c084eb7-1091-4bfa-a980-dca0e80195ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.810267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084eb7-1091-4bfa-a980-dca0e80195ca admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.810555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084eb7-1091-4bfa-a980-dca0e80195ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.810833 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c084eb7-1091-4bfa-a980-dca0e80195ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.812121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084eb7-1091-4bfa-a980-dca0e80195ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.813729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c084eb7-1091-4bfa-a980-dca0e80195ca admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.824151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1373/2762] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:23.829187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:23.840644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2ac46-7279-44c8-a943-b40fd6fe3e61 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:23.845809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:23.846853 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.847179 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.847558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] PATH_INFO: `/v3/projects/34566d45a4844c2abea583db2e3f747a/users/16445633dee2446baa66f4da7de5de78/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.863083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34566d45a4844c2abea583db2e3f747a, user_id=16445633dee2446baa66f4da7de5de78, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:23.865848 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:23.879992 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1390/2763] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:23 2022] PUT /identity/v3/projects/34566d45a4844c2abea583db2e3f747a/users/16445633dee2446baa66f4da7de5de78/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:23.892403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:23.892727 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:23.893018 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:23.923555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] MFA Rules not processed for user `16445633dee2446baa66f4da7de5de78`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:23.925857 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:24.012440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1374/2764] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:24.024943 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:24.026851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:24.086736 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74b74c-7313-4777-abfb-586cfc0545d8 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:24.090475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3278c-80be-4b7a-a5f9-1437dc763f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:24.091124 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3278c-80be-4b7a-a5f9-1437dc763f1d service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:24.091408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3278c-80be-4b7a-a5f9-1437dc763f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:24.091687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3278c-80be-4b7a-a5f9-1437dc763f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:24.101024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3278c-80be-4b7a-a5f9-1437dc763f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:24.102533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3278c-80be-4b7a-a5f9-1437dc763f1d service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:24.112228 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1391/2765] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:24.131971 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:24.196782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f897ef3-90d4-4c93-9213-399e8a783df9 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:24.201384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:24.202397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:24.202923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:24.203219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] PATH_INFO: `/v3/users/16445633dee2446baa66f4da7de5de78` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:24.204484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=16445633dee2446baa66f4da7de5de78)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:24.206444 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:24.220692 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:24.255978 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16445633dee2446baa66f4da7de5de78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.263152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16445633dee2446baa66f4da7de5de78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.263765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16445633dee2446baa66f4da7de5de78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.264402 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16445633dee2446baa66f4da7de5de78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.272389 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16445633dee2446baa66f4da7de5de78'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.291599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1375/2766] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:04:24 2022] DELETE /identity/v3/users/16445633dee2446baa66f4da7de5de78 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:04:24.298497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:24.299259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:24.299259 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:24.318108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:24.319301 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:24.390656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1392/2767] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:24.401496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:24.403103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:24.470934 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036dbfd-83db-4230-bc81-bcfcf260b0cd admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:24.475439 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe1588e-2f3a-49b6-852a-f6f7f23a1d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:24.476187 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1588e-2f3a-49b6-852a-f6f7f23a1d4b service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:24.476475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1588e-2f3a-49b6-852a-f6f7f23a1d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:24.476836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe1588e-2f3a-49b6-852a-f6f7f23a1d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:24.487291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1588e-2f3a-49b6-852a-f6f7f23a1d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:24.488817 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe1588e-2f3a-49b6-852a-f6f7f23a1d4b service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:24.499881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1376/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:04:24.803734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:24.805201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:24.819076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e7a4b-7648-4c5d-929d-bfb0ff6533a2 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:24.823733 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:24.825100 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:24.825393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:24.825675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] PATH_INFO: `/v3/projects/34566d45a4844c2abea583db2e3f747a` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:24.827170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34566d45a4844c2abea583db2e3f747a)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:24.829687 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:24.876097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:24.877739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34566d45a4844c2abea583db2e3f747a', 'request_id': 'req-3542361b-9360-435a-be3f-cc6315cf7e15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.892034 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34566d45a4844c2abea583db2e3f747a', 'request_id': 'req-3542361b-9360-435a-be3f-cc6315cf7e15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.900165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34566d45a4844c2abea583db2e3f747a', 'request_id': 'req-3542361b-9360-435a-be3f-cc6315cf7e15'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.912765 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] The token cache is being invalidate because project 34566d45a4844c2abea583db2e3f747a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:04:24.913119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.913866 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3542361b-9360-435a-be3f-cc6315cf7e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:04:24.915898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1393/2769] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:04:24 2022] DELETE /identity/v3/projects/34566d45a4844c2abea583db2e3f747a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:24.921277 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:24.921588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:24.921588 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:24.939622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:24.940864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:25.003499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1377/2770] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:25.011634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.011924 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.012254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.030952 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:25.032183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:25.040140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1394/2771] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:25.044423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.059168 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa703-3451-4a5a-939b-11ce63627514 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.064632 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.065285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.065565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.065838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.066561 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:04:25.066861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.068496 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.080222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1378/2772] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:04:25.085020 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.096541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25068fd3-46ca-4b49-82c8-0233fdb2ea97 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.101312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.101965 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.102247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.102529 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.103225 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.105189 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.137818 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1395/2773] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:04:25 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:25.138957 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.149050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfddc31-691e-4038-ab0d-729f8bead3bf admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.168885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:25.209968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.210700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.210987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.211268 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.211961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.213917 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.234542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1379/2774] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:04:25 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:04:25.238357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.248814 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5363c770-fa3d-43d2-a55f-b0167718caa6 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.254224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.254905 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.255187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.255459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.256211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.257809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.268363 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1396/2775] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:04:25.272271 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.282629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa09580-3b9f-4310-8060-cb9a3b805db6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.287188 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e28c8-e269-4521-9feb-7fbb6ae46943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.287859 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e28c8-e269-4521-9feb-7fbb6ae46943 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.288171 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e28c8-e269-4521-9feb-7fbb6ae46943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.288450 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e28c8-e269-4521-9feb-7fbb6ae46943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.289195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e28c8-e269-4521-9feb-7fbb6ae46943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.290777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e28c8-e269-4521-9feb-7fbb6ae46943 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.301234 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1380/2776] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:04:25.305313 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.315476 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed9e08-ecca-492b-a3af-a81b6188534c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.320209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.321089 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.321417 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.321651 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] PATH_INFO: `/v3/projects/be41bb45510e4ac0b1fb096fcf6f9417/users/5307f0919a1e4bb1bfd0e674891dbd77/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.336443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be41bb45510e4ac0b1fb096fcf6f9417, user_id=5307f0919a1e4bb1bfd0e674891dbd77, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.339119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.353306 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1397/2777] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:04:25 2022] PUT /identity/v3/projects/be41bb45510e4ac0b1fb096fcf6f9417/users/5307f0919a1e4bb1bfd0e674891dbd77/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:04:25.364823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.365155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.365474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.393121 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] MFA Rules not processed for user `5307f0919a1e4bb1bfd0e674891dbd77`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:25.394280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:25.458156 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1381/2778] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:25.470550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:25.472261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:25.529827 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f191eabd-01b9-400a-9330-f1fa335855b3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:25.534508 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846bac7d-4ed1-46b0-a2fd-8c028a609644 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:25.535156 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bac7d-4ed1-46b0-a2fd-8c028a609644 service glance] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:25.535443 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bac7d-4ed1-46b0-a2fd-8c028a609644 service glance] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:25.535726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846bac7d-4ed1-46b0-a2fd-8c028a609644 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:25.544055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bac7d-4ed1-46b0-a2fd-8c028a609644 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:25.545540 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846bac7d-4ed1-46b0-a2fd-8c028a609644 service glance] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:25.555146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1398/2779] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:30.825606 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:04:30.827033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:30.916446 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a93f3-7952-45c8-9374-a1e2a9ff6527 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:04:30.921555 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:04:30.922172 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:30.922460 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:30.922735 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:30.924542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:30.938249 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:04:30.940286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:04:30.951738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1382/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:04:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:04:32.904227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27421256-fa6b-439f-b269-209c2569b751 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:04:32.904849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27421256-fa6b-439f-b269-209c2569b751 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:04:32.904849 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27421256-fa6b-439f-b269-209c2569b751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:04:32.921805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-27421256-fa6b-439f-b269-209c2569b751 None None] MFA Rules not processed for user `0699a27504da472e96b2dbd52d6cf301`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:04:32.923031 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-27421256-fa6b-439f-b269-209c2569b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:04:32.979826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1399/2781] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:04:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:04:40.948330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:04:42.973013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:01.941870 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:01.943466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:01.998854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23868c1-71df-49e4-9717-214eb1fb42e2 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:02.002553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:02.003204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:02.003492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:02.003772 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:02.005745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:02.078388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:02.079825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:02.091308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1383/2782] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:05:03.275519 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27421256-fa6b-439f-b269-209c2569b751 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:03.277050 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-27421256-fa6b-439f-b269-209c2569b751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:03.290473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27421256-fa6b-439f-b269-209c2569b751 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:03.294013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:03.294655 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:03.294939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:03.295216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:03.296981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:03.310808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:03.312231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c64f66-266b-469d-9112-56a85f48fe46 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:03.325118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1400/2783] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:05:04.844348 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:04.907347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b546eca-9374-4c02-90f3-ff2e71fc5111 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:04.911986 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:04.912946 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:04.913230 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:04.913537 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] PATH_INFO: `/v3/users/5307f0919a1e4bb1bfd0e674891dbd77` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:04.914733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5307f0919a1e4bb1bfd0e674891dbd77)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:04.916723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:04.960752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:04.963462 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5307f0919a1e4bb1bfd0e674891dbd77'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:04.970837 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5307f0919a1e4bb1bfd0e674891dbd77'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:04.971566 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5307f0919a1e4bb1bfd0e674891dbd77'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:04.972216 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5307f0919a1e4bb1bfd0e674891dbd77'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:04.979445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5307f0919a1e4bb1bfd0e674891dbd77'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:04.997721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1384/2784] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:05:04 2022] DELETE /identity/v3/users/5307f0919a1e4bb1bfd0e674891dbd77 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:05:05.002302 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.002776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.002874 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.022429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:05.023644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:05.084140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1401/2785] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:05.093850 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.095257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:05.152995 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d4234-b860-49c7-9e4a-57e3db43d9e5 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.156694 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01204f1-1a90-40bb-b796-d32934c153de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.157344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01204f1-1a90-40bb-b796-d32934c153de service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.157628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01204f1-1a90-40bb-b796-d32934c153de service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.157913 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01204f1-1a90-40bb-b796-d32934c153de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.167157 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01204f1-1a90-40bb-b796-d32934c153de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.168636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01204f1-1a90-40bb-b796-d32934c153de service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.178327 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1385/2786] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:05:05.451083 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.452574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:05.466220 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910abd1-40c4-436f-9851-464797de45a4 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.471277 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.472182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.472463 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.472767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] PATH_INFO: `/v3/projects/be41bb45510e4ac0b1fb096fcf6f9417` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.474671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be41bb45510e4ac0b1fb096fcf6f9417)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.476726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.525789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be41bb45510e4ac0b1fb096fcf6f9417', 'request_id': 'req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:05.533907 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be41bb45510e4ac0b1fb096fcf6f9417', 'request_id': 'req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:05.543504 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be41bb45510e4ac0b1fb096fcf6f9417', 'request_id': 'req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:05.554732 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] The token cache is being invalidate because project be41bb45510e4ac0b1fb096fcf6f9417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:05:05.555021 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:05.555659 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-eab30620-89ad-4a7e-b9ed-66b1e416c7b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:05.557229 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1402/2787] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:05:05 2022] DELETE /identity/v3/projects/be41bb45510e4ac0b1fb096fcf6f9417 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:05:05.562225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.562519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.562809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.580109 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:05.581339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:05.642572 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1386/2788] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:05.652258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.652258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.652258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.673357 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:05.674570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:05.683339 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1403/2789] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:05.687681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.698809 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b560-421f-414f-87ad-ba0450617e00 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.703987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.704947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.704947 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.705228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.706035 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:05:05.706267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.707887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.721295 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1387/2790] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:05:05.725771 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.735294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4164b9-3db1-4c2d-bb2d-a1c76ac7f9dc None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.743396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.743396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.743396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.743396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.744067 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.744589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.783452 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1404/2791] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:05:05 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:05:05.784861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.794760 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0504e1d-9d0c-4d57-8a8e-8a9b0cd72019 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.842095 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:05.854625 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.855485 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.855770 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.856048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.856789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.858745 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.878467 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1388/2792] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:05:05 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:05:05.883586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.894813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449469e-6315-4d82-8119-831aa068d365 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.900320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.901035 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.901325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.901604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.902427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.904054 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.915929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1405/2793] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:05:05.919843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.930048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410b9a2-094f-4493-865f-300f5fd9b0eb admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.934903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b954d9-9317-435d-aaa8-3bde87a70120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.935551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b954d9-9317-435d-aaa8-3bde87a70120 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.935834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b954d9-9317-435d-aaa8-3bde87a70120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.936148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b954d9-9317-435d-aaa8-3bde87a70120 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.936881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b954d9-9317-435d-aaa8-3bde87a70120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.938498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b954d9-9317-435d-aaa8-3bde87a70120 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:05.949403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1389/2794] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:05:05.953589 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:05.964837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc081f-33d8-4844-ac0c-112bb400da0b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:05.969388 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:05.970269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:05.970546 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:05.970815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] PATH_INFO: `/v3/projects/377898fbfd124ac2b24c7da71aac535f/users/97615081277747a6b17b51df49fc2a85/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:05.985269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=377898fbfd124ac2b24c7da71aac535f, user_id=97615081277747a6b17b51df49fc2a85, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:05.987922 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:06.001585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1406/2795] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:05:05 2022] PUT /identity/v3/projects/377898fbfd124ac2b24c7da71aac535f/users/97615081277747a6b17b51df49fc2a85/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:05:06.012677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:06.012969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:06.013248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:06.038874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] MFA Rules not processed for user `97615081277747a6b17b51df49fc2a85`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:06.040058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:06.105441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1390/2796] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:06.117266 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:06.118989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:06.129217 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:06.184207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a59e76-d4ae-4c4b-9710-87c5f3f84c0d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:06.187769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:06.188448 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:06.188729 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:06.189004 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:06.198473 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:06.200000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:06.208930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1407/2797] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:05:09.510130 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:09.511662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:09.526046 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc343d0-7257-44bb-b0e0-61e8245b5b0a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:09.530245 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:09.530937 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:09.531221 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:09.531500 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:09.533296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:09.617961 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:09.619685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:09.628902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1391/2798] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:05:16.204808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=23 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:19.624583 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:21.130904 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:21.132551 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:21.211332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae18a3-a27b-47e2-836f-187fa616b835 service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:21.215466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:21.216119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:21.216411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:21.216690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:21.218573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:21.229895 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:21.231364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:21.241214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1408/2799] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:05:21.352682 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:21.361911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c95b0d9-7e34-464b-975f-c32e4eeb00fb service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:21.365648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:21.366590 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:21.366882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:21.367164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:21.368997 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:21.444489 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:21.446293 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:21.457628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1392/2800] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:05:31.236414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=26 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:31.452574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=34 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:38.736433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:38.737860 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:38.768210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7dd4c-4f7a-4c28-8834-e71f0e9c3e24 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:38.772103 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:38.772794 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:38.773073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:38.773369 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:38.775188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:38.786056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:38.787418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91508a64-d4b2-4fe1-babb-3dee33a5e029 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:38.796042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1409/2801] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:05:48.791780 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=15 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:49.771895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:49.837894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef714aae-a93d-4c04-a197-f88c456d79a5 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:49.842472 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:49.843338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:49.843618 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:49.843893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] PATH_INFO: `/v3/users/97615081277747a6b17b51df49fc2a85` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:49.845124 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97615081277747a6b17b51df49fc2a85)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:49.847125 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:49.895506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97615081277747a6b17b51df49fc2a85'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:49.902782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97615081277747a6b17b51df49fc2a85'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:49.903397 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97615081277747a6b17b51df49fc2a85'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:49.904011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97615081277747a6b17b51df49fc2a85'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:49.911596 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97615081277747a6b17b51df49fc2a85'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:49.929651 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1393/2802] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:05:49 2022] DELETE /identity/v3/users/97615081277747a6b17b51df49fc2a85 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:05:49.934496 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176676a-7750-4af4-973b-438339606b11 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:49.934976 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176676a-7750-4af4-973b-438339606b11 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:49.935079 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176676a-7750-4af4-973b-438339606b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:49.952813 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-7176676a-7750-4af4-973b-438339606b11 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:49.953995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7176676a-7750-4af4-973b-438339606b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:50.018418 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1410/2803] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:50.028487 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.029891 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:50.091708 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e259ef8-628f-4cb4-89c0-270fe55cae26 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.095326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f82701-47d4-4291-9f7a-3b1be5b10ee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.095979 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82701-47d4-4291-9f7a-3b1be5b10ee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.096302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82701-47d4-4291-9f7a-3b1be5b10ee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.097214 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f82701-47d4-4291-9f7a-3b1be5b10ee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.106170 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f82701-47d4-4291-9f7a-3b1be5b10ee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.107602 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f82701-47d4-4291-9f7a-3b1be5b10ee0 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.117910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1394/2804] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:05:50.388173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176676a-7750-4af4-973b-438339606b11 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.389726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-7176676a-7750-4af4-973b-438339606b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:50.401614 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176676a-7750-4af4-973b-438339606b11 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.406483 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.407303 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.407585 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.407864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] PATH_INFO: `/v3/projects/377898fbfd124ac2b24c7da71aac535f` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.409253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=377898fbfd124ac2b24c7da71aac535f)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.411212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.463929 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '377898fbfd124ac2b24c7da71aac535f', 'request_id': 'req-bbe62299-5b7f-4060-b17a-6d4fd6981874'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:50.471014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '377898fbfd124ac2b24c7da71aac535f', 'request_id': 'req-bbe62299-5b7f-4060-b17a-6d4fd6981874'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:50.477648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '377898fbfd124ac2b24c7da71aac535f', 'request_id': 'req-bbe62299-5b7f-4060-b17a-6d4fd6981874'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:50.488763 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] The token cache is being invalidate because project 377898fbfd124ac2b24c7da71aac535f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:05:50.489053 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:50.489714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-bbe62299-5b7f-4060-b17a-6d4fd6981874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:05:50.491234 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1411/2805] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:05:50 2022] DELETE /identity/v3/projects/377898fbfd124ac2b24c7da71aac535f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:05:50.496530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.496823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.497103 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.516573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:50.517765 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:50.529464 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:50.579598 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1395/2806] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:05:50.587863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.588192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.588487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.608565 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:50.609968 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:50.619995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1412/2807] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:50.624287 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.635393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf87efd4-56a8-46f5-a01f-2b4b7bd12b0e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.640678 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.641440 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.641816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.642173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.643126 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:05:50.643611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.645267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.658097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1396/2808] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:05:50.663091 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.672844 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207557a-4770-4b61-9795-3d366f3a85aa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.678347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.678997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.679281 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.679561 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.680284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.682209 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.708811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1413/2809] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:05:50 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:05:50.712966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.723663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b389826b-9ef1-4ade-88c9-0ae2de7f322b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.782667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.783344 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.783630 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.784498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.785447 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.788228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.813351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1397/2810] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:05:50 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:05:50.818134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.829890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78813d6b-c532-44c4-9b6b-e99a26a5ef1e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.835223 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.835879 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.836194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.836477 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.837212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.838833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.850730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1414/2811] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:05:50.854758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.865925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0acef05-f4dd-4749-b924-4ac2811a8f21 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.870838 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a7185-d14a-4794-a4ae-b29a6345325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.871493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7185-d14a-4794-a4ae-b29a6345325c admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.871782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7185-d14a-4794-a4ae-b29a6345325c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.872374 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a7185-d14a-4794-a4ae-b29a6345325c admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.873106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a7185-d14a-4794-a4ae-b29a6345325c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.874713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a7185-d14a-4794-a4ae-b29a6345325c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.885276 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1398/2812] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:05:50.889289 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:50.900121 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d200d07-337d-4c63-9c55-2d09d85d8447 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:50.904761 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:50.905636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.905923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.906204 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] PATH_INFO: `/v3/projects/c20856b8ccd841f79e28f9c94a097d8b/users/131f2bda5b2d4566a3ccdd2ed6766bb9/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.922350 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20856b8ccd841f79e28f9c94a097d8b, user_id=131f2bda5b2d4566a3ccdd2ed6766bb9, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:50.926231 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:50.938877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1415/2813] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:05:50 2022] PUT /identity/v3/projects/c20856b8ccd841f79e28f9c94a097d8b/users/131f2bda5b2d4566a3ccdd2ed6766bb9/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:05:50.949503 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:50.949798 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:50.950079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:50.978003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] MFA Rules not processed for user `131f2bda5b2d4566a3ccdd2ed6766bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:05:50.979118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:51.048463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1399/2814] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:05:51.060951 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:05:51.062523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:05:51.086563 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:05:51.125724 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9602d-428e-4e14-86f7-180061d57f4b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:05:51.130139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1666e2af-2308-4e16-9e28-41bb309ed1bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:05:51.130798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666e2af-2308-4e16-9e28-41bb309ed1bf service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:05:51.131081 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666e2af-2308-4e16-9e28-41bb309ed1bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:05:51.131351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1666e2af-2308-4e16-9e28-41bb309ed1bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:05:51.140033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666e2af-2308-4e16-9e28-41bb309ed1bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:05:51.141491 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1666e2af-2308-4e16-9e28-41bb309ed1bf service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:05:51.150739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1416/2815] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:05:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:06:00.999573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:01.016015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:01.068717 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c9ae4-466f-4e10-8c02-66fb3e5be526 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:01.074925 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:01.075834 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.076129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.076414 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] PATH_INFO: `/v3/users/131f2bda5b2d4566a3ccdd2ed6766bb9` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.077658 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=131f2bda5b2d4566a3ccdd2ed6766bb9)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:01.082471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:01.133877 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '131f2bda5b2d4566a3ccdd2ed6766bb9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.141411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '131f2bda5b2d4566a3ccdd2ed6766bb9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.142177 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '131f2bda5b2d4566a3ccdd2ed6766bb9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.143060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '131f2bda5b2d4566a3ccdd2ed6766bb9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.146703 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=17 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:01.151779 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '131f2bda5b2d4566a3ccdd2ed6766bb9'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.170073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1400/2816] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:06:00 2022] DELETE /identity/v3/users/131f2bda5b2d4566a3ccdd2ed6766bb9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:06:01.174631 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.174966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.175372 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.193106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:01.194360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:01.254402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1417/2817] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:01.264139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:01.265563 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:01.324800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86421be6-7f3c-49c1-bcae-a50ce8a512b8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:01.328510 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a3307f-a9aa-48c6-b297-8c7219478766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:01.329158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3307f-a9aa-48c6-b297-8c7219478766 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.329445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3307f-a9aa-48c6-b297-8c7219478766 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.329730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a3307f-a9aa-48c6-b297-8c7219478766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.339072 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3307f-a9aa-48c6-b297-8c7219478766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:01.340551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a3307f-a9aa-48c6-b297-8c7219478766 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:01.350865 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1401/2818] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:06:01.638059 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:01.639826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:01.656488 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdc3a8f-d584-4aa0-a4fd-03c2b4c39d6e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:01.665799 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:01.667152 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.667586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.668108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] PATH_INFO: `/v3/projects/c20856b8ccd841f79e28f9c94a097d8b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.669795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20856b8ccd841f79e28f9c94a097d8b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:01.672304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:01.727558 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20856b8ccd841f79e28f9c94a097d8b', 'request_id': 'req-3ab73479-2eeb-4134-a036-0d6db1b8fed4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.744157 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20856b8ccd841f79e28f9c94a097d8b', 'request_id': 'req-3ab73479-2eeb-4134-a036-0d6db1b8fed4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.755422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20856b8ccd841f79e28f9c94a097d8b', 'request_id': 'req-3ab73479-2eeb-4134-a036-0d6db1b8fed4'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.766106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] The token cache is being invalidate because project c20856b8ccd841f79e28f9c94a097d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:06:01.766305 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.767002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-3ab73479-2eeb-4134-a036-0d6db1b8fed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:06:01.773701 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1418/2819] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:06:01 2022] DELETE /identity/v3/projects/c20856b8ccd841f79e28f9c94a097d8b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:06:01.773989 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.774576 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.775154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.794635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:01.795848 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:01.828546 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:01.860699 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1402/2820] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:01.869466 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.869757 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.870042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.888307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:01.889509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:01.898174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1419/2821] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:01.902769 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:01.913340 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7800daa-defc-41b1-bf27-88cdb2e74a17 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:01.917964 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:01.918620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.918909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.919191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.919928 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:06:01.920275 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:01.921894 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:01.934771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1403/2822] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:06:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:06:01.939309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:01.952842 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47327f-a514-42dc-89e1-439325eee7ac None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:01.958174 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:01.958832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:01.959114 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:01.959393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:01.960112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:01.962055 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:01.988052 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1420/2823] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:06:01 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:06:01.992573 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:02.003062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2281a0-13bf-4b9f-a8e6-998022efec26 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:02.060036 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:02.060663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:02.060952 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:02.061233 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:02.061944 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:02.063896 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:02.085484 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1404/2824] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:06:01 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:06:02.089766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:02.100749 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf68ba10-0640-4e24-a9c4-b9700cce04f5 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:02.109462 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:02.110142 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:02.110428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:02.110708 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:02.111455 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:02.113280 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:02.125147 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1421/2825] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:06:02.130666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:02.143778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba058b30-4c83-4392-b897-04fd4e3c2969 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:02.150863 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3ce70-cbcd-4f0d-a9f3-fcb8db5dbbe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:02.151599 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3ce70-cbcd-4f0d-a9f3-fcb8db5dbbe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:02.151936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3ce70-cbcd-4f0d-a9f3-fcb8db5dbbe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:02.152309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3ce70-cbcd-4f0d-a9f3-fcb8db5dbbe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:02.153128 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3ce70-cbcd-4f0d-a9f3-fcb8db5dbbe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:02.154987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3ce70-cbcd-4f0d-a9f3-fcb8db5dbbe1 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:02.168737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1405/2826] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:06:02.173396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:02.184908 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67a164-1342-41b5-a503-366be3b53314 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:02.189583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:02.190470 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:02.190779 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:02.191068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] PATH_INFO: `/v3/projects/c128406f10d844baa248e415f8169026/users/afd610c316f644b68fb1b61824384f01/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:02.206894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c128406f10d844baa248e415f8169026, user_id=afd610c316f644b68fb1b61824384f01, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:02.209711 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:02.224207 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1422/2827] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:06:02 2022] PUT /identity/v3/projects/c128406f10d844baa248e415f8169026/users/afd610c316f644b68fb1b61824384f01/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:06:02.229378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296df8-5dde-4d1e-85cd-0286f949034a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:02.229777 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296df8-5dde-4d1e-85cd-0286f949034a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:02.230152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296df8-5dde-4d1e-85cd-0286f949034a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:02.248705 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-a0296df8-5dde-4d1e-85cd-0286f949034a None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:02.249941 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a0296df8-5dde-4d1e-85cd-0286f949034a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:02.435304 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1406/2828] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:06:02.446078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:02.448045 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:02.476858 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:02.517667 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad1bed-5f35-4499-ab7f-0b4df481da5e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:02.522201 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:02.522851 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:02.523136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:02.523415 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:02.532672 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:02.534123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:02.544379 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1423/2829] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:06:07.289453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.290031 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.290502 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.336987 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] MFA Rules not processed for user `afd610c316f644b68fb1b61824384f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:07.339229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:07.376895 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:07.418844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1407/2830] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:07.423823 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:07.438344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fc40-777a-45b6-8548-cebd194c5dcc service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:07.444416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:07.445183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.445595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.446001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.446900 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:07.450108 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:07.482171 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1424/2831] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:06:07 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:06:07.486490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:07.501885 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9e0548-3bd1-42d7-b1c2-a2f8ff23f34a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:07.575733 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:07.576505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.576920 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.577302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.578132 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:07.580273 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:07.601685 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1408/2832] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:06:07 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:06:07.605932 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:07.617695 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3f81c-52bc-4ba0-9f35-3a1f40482400 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:07.622811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:07.623471 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.623754 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.624032 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.624795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:07.626412 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:07.638805 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1425/2833] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:06:07.643145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:07.653919 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039fde7-6073-453a-99d5-6f3e959530ff admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:07.658501 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:07.660074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.660074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.660074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] PATH_INFO: `/v3/projects/0362a5b1fb294744bd25f50c3c0e5a67/users/752da97b9da945459a09a51b626f73f2/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.677073 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] RBAC: Authorizing `identity:create_grant(project_id=0362a5b1fb294744bd25f50c3c0e5a67, user_id=752da97b9da945459a09a51b626f73f2, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:07.679749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:07.697752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1409/2834] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:06:07 2022] PUT /identity/v3/projects/0362a5b1fb294744bd25f50c3c0e5a67/users/752da97b9da945459a09a51b626f73f2/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:06:07.702325 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:07.713903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f2f691-28d6-4b0f-b024-34ca41d11230 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:07.787422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797cfa7-68a4-4022-ac53-2ad4c2dffe0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:07.788509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797cfa7-68a4-4022-ac53-2ad4c2dffe0c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.788509 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797cfa7-68a4-4022-ac53-2ad4c2dffe0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.788737 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797cfa7-68a4-4022-ac53-2ad4c2dffe0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.789406 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797cfa7-68a4-4022-ac53-2ad4c2dffe0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:07.791036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797cfa7-68a4-4022-ac53-2ad4c2dffe0c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:07.801821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1426/2835] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:06:07.806229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:07.817388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670ff89-f534-4204-a292-bdf3eca027da admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:07.822054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:07.822942 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:07.823223 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:07.823493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] PATH_INFO: `/v3/projects/0362a5b1fb294744bd25f50c3c0e5a67/users/752da97b9da945459a09a51b626f73f2/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:07.825560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0362a5b1fb294744bd25f50c3c0e5a67, user_id=752da97b9da945459a09a51b626f73f2, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:07.828298 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:07.839549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1410/2836] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:06:07 2022] PUT /identity/v3/projects/0362a5b1fb294744bd25f50c3c0e5a67/users/752da97b9da945459a09a51b626f73f2/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:06:12.569501 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2530-2733-47f7-8c6b-34b6427d34ef None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:12.569985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2530-2733-47f7-8c6b-34b6427d34ef None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:12.570136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d2530-2733-47f7-8c6b-34b6427d34ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:12.603570 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-299d2530-2733-47f7-8c6b-34b6427d34ef None None] MFA Rules not processed for user `752da97b9da945459a09a51b626f73f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:12.605125 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-299d2530-2733-47f7-8c6b-34b6427d34ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:12.693782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1427/2837] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:12.707740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:12.710256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:12.778318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf8e23-8b85-4cea-83cb-c5f6959cce7d admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:12.782285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:12.782939 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:12.783227 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:12.783506 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:12.829758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:12.859097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:12.860629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:12.870906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1411/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:06:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:06:12.971036 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae15e31-1cff-4ffd-a04e-26b14b818561 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:12.971333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae15e31-1cff-4ffd-a04e-26b14b818561 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:12.972600 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae15e31-1cff-4ffd-a04e-26b14b818561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:12.993840 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-fae15e31-1cff-4ffd-a04e-26b14b818561 None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:12.995097 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-fae15e31-1cff-4ffd-a04e-26b14b818561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:13.000838 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:13.060808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1428/2839] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:13.110074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:13.111535 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:13.186893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ebf43-e7b9-4c73-9187-4aa2f58bf59f service glance] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:13.192151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaac348-3f4c-438f-b0f0-c254b3e4e3f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:13.192811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac348-3f4c-438f-b0f0-c254b3e4e3f8 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:13.193100 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac348-3f4c-438f-b0f0-c254b3e4e3f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:13.193383 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac348-3f4c-438f-b0f0-c254b3e4e3f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:13.201753 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac348-3f4c-438f-b0f0-c254b3e4e3f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:13.203354 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac348-3f4c-438f-b0f0-c254b3e4e3f8 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:13.211575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1412/2840] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 20:06:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 20:06:22.966837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:22.967408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:22.967408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:22.984789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] MFA Rules not processed for user `afd610c316f644b68fb1b61824384f01`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:22.985999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:22.992854 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1429/2841] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:06:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:23.208562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=42 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:24.841050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:24.841624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:24.841624 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:24.859860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:06:24.860633 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:24.873511 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1413/2842] 10.222.0.17 () {58 vars in 1009 bytes} [Fri Jul 15 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:06:24.882271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:24.895261 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03271cc6-339c-42f2-93a8-466d5e32fa2a None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:24.900246 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:24.900897 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:24.901186 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:24.901469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:24.911173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:24.912821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:24.921317 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1430/2843] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 20:06:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 15 20:06:34.853954 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:34.917808 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=29 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:37.698613 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:37.700129 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:37.813251 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d1699-9760-4f04-add0-a130de81fe5a None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:37.822933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:37.824910 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:37.825474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:37.825883 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:37.828764 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:37.843423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:37.845065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:37.857205 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1414/2844] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:06:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:06:47.852317 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=34 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:06:57.397341 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:57.398811 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:57.454960 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577e911-6016-4a35-8785-3d372def2440 service swift] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:57.459404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:57.460044 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:57.460356 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:57.460634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:57.469497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:57.470933 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:57.480569 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1431/2845] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:06:58.359718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:06:58.369963 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5007f33d-3c0b-404d-88be-6414d5d2eb24 service nova] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:06:58.373691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:06:58.374508 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:06:58.374936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:06:58.375285 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:06:58.377311 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:06:58.448152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:06:58.449620 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:06:58.459456 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1415/2846] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:06:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:01.714409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:01.733980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaf645-fac0-4bab-a8fc-84e315c04a79 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:01.747139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:01.749746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:01.750043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:01.750316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:01.762901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:01.794328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:01.796014 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:01.823609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1432/2847] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:02.990912 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:02.992680 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:03.007533 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a94d0a-df4c-475f-ba80-cefde1ce4c41 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.012801 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.013726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.014005 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.014371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] PATH_INFO: `/v3/users/837f13aef58b455e95372c6f106d5807` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.015773 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837f13aef58b455e95372c6f106d5807)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.017864 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.064134 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837f13aef58b455e95372c6f106d5807'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.071935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837f13aef58b455e95372c6f106d5807'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.072612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837f13aef58b455e95372c6f106d5807'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.073289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837f13aef58b455e95372c6f106d5807'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.080799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-6cc10085-6b6b-4a8c-8d52-8272e1b89b24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837f13aef58b455e95372c6f106d5807'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.103288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1416/2848] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:02 2022] DELETE /identity/v3/users/837f13aef58b455e95372c6f106d5807 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:03.279192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.291001 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed8113-42ca-45d4-a1a3-02d3ad805c32 service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.349953 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.350817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.351098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.351373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] PATH_INFO: `/v3/projects/e39f25de38b1465784de70ebf4e930a9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.362023 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e39f25de38b1465784de70ebf4e930a9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.364689 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.411649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e39f25de38b1465784de70ebf4e930a9', 'request_id': 'req-958b0f36-b529-45fc-a1d3-fed2f3edec32'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.418995 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e39f25de38b1465784de70ebf4e930a9', 'request_id': 'req-958b0f36-b529-45fc-a1d3-fed2f3edec32'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.426333 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e39f25de38b1465784de70ebf4e930a9', 'request_id': 'req-958b0f36-b529-45fc-a1d3-fed2f3edec32'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.437762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] The token cache is being invalidate because project e39f25de38b1465784de70ebf4e930a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:03.438046 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.438680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-958b0f36-b529-45fc-a1d3-fed2f3edec32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:03.440332 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1433/2849] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:03 2022] DELETE /identity/v3/projects/e39f25de38b1465784de70ebf4e930a9 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:03.446350 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.446663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.446945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.465291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:03.466498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:03.532068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1417/2850] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:03.540575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.540863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.541136 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.561728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:03.563173 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:03.572938 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1434/2851] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:03.577676 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.587532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce020810-e688-48da-887a-db960e55d2c1 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.592105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.592756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.593065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.593310 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.594050 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:07:03.594422 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.595922 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.598025 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:03.610099 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1418/2852] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:07:03.614804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.620776 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:03.625104 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73267b5d-1421-4aa2-97b5-c8c494175fd5 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.630999 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.631643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.632146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.632604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.633606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.635438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.661200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1435/2853] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:07:03 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:07:03.666010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.677729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679b7ea0-ddac-4de7-974f-486894098968 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.735613 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.736313 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.736597 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.736872 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.737565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.743704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.765090 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1419/2854] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:07:03 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:03.770127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.784497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cccffe-dfdd-493b-826f-5157ab9f7565 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.790351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.791026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.791310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.791590 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.792362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.793982 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.809588 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1436/2855] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:03.810611 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.822286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e0943-e611-4aa6-a4bd-9178310e1fd6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.827844 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ae6df-afa1-4186-9fbf-84f5fa1ca655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.828628 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ae6df-afa1-4186-9fbf-84f5fa1ca655 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.828943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ae6df-afa1-4186-9fbf-84f5fa1ca655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.829246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ae6df-afa1-4186-9fbf-84f5fa1ca655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.830054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ae6df-afa1-4186-9fbf-84f5fa1ca655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.831982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ae6df-afa1-4186-9fbf-84f5fa1ca655 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.842544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1420/2856] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:07:03.846603 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:03.856669 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a5e1f6-a486-45cd-a53a-ff4e08dc385b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:03.861335 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:03.862310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.862622 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.862930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] PATH_INFO: `/v3/projects/b2d0730e64f647e6a53521fef94a887c/users/47e0fbd7f5ee4d09a1aa875a489fdf27/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.879973 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2d0730e64f647e6a53521fef94a887c, user_id=47e0fbd7f5ee4d09a1aa875a489fdf27, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:03.882674 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:03.895362 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1437/2857] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:03 2022] PUT /identity/v3/projects/b2d0730e64f647e6a53521fef94a887c/users/47e0fbd7f5ee4d09a1aa875a489fdf27/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:03.906119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:03.906413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:03.906729 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:03.933220 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] MFA Rules not processed for user `47e0fbd7f5ee4d09a1aa875a489fdf27`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:03.934356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:04.001899 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1421/2858] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:04.013751 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:04.015459 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:04.088872 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4ea9d-76a1-4bf4-a6fd-cef71159f87e admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:04.094420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44907269-7fde-4a7c-8c19-ad8380f2f7f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:04.095072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44907269-7fde-4a7c-8c19-ad8380f2f7f9 service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:04.095360 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44907269-7fde-4a7c-8c19-ad8380f2f7f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:04.095637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44907269-7fde-4a7c-8c19-ad8380f2f7f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:04.104739 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44907269-7fde-4a7c-8c19-ad8380f2f7f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:04.106358 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44907269-7fde-4a7c-8c19-ad8380f2f7f9 service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:04.117127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1438/2859] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:04.346873 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:04.414394 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837fdab-7623-4d02-87ea-281356e2f082 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:04.418888 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:04.419750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:04.420033 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:04.420336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] PATH_INFO: `/v3/users/47e0fbd7f5ee4d09a1aa875a489fdf27` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:04.421517 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47e0fbd7f5ee4d09a1aa875a489fdf27)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:04.423445 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:04.426706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:04.475054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47e0fbd7f5ee4d09a1aa875a489fdf27'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:04.482386 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47e0fbd7f5ee4d09a1aa875a489fdf27'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:04.483058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47e0fbd7f5ee4d09a1aa875a489fdf27'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:04.483679 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47e0fbd7f5ee4d09a1aa875a489fdf27'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:04.491308 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47e0fbd7f5ee4d09a1aa875a489fdf27'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:04.515936 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1422/2860] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:04 2022] DELETE /identity/v3/users/47e0fbd7f5ee4d09a1aa875a489fdf27 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:04.523598 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:04.524163 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:04.524593 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:04.548145 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:04.549348 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:04.610238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1439/2861] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:04.621265 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:04.622768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:04.682175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6ce9e-927c-4a72-b911-7e84ddc18458 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:04.685861 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c396d1ea-0b74-4ca2-8cf8-390f1ac4c6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:04.686513 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396d1ea-0b74-4ca2-8cf8-390f1ac4c6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:04.686799 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396d1ea-0b74-4ca2-8cf8-390f1ac4c6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:04.687089 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c396d1ea-0b74-4ca2-8cf8-390f1ac4c6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:04.696868 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396d1ea-0b74-4ca2-8cf8-390f1ac4c6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:04.698321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c396d1ea-0b74-4ca2-8cf8-390f1ac4c6c8 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:04.709065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1423/2862] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:05.009912 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.011534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:05.025058 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c36118-2998-4a1d-ae7a-891570f33d86 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.029817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.030714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.030974 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.031254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] PATH_INFO: `/v3/projects/b2d0730e64f647e6a53521fef94a887c` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.032487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2d0730e64f647e6a53521fef94a887c)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.034421 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.071090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:05.089613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2d0730e64f647e6a53521fef94a887c', 'request_id': 'req-6d5a84cc-aabb-4534-a46a-0315a7572540'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:05.097056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2d0730e64f647e6a53521fef94a887c', 'request_id': 'req-6d5a84cc-aabb-4534-a46a-0315a7572540'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:05.105662 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2d0730e64f647e6a53521fef94a887c', 'request_id': 'req-6d5a84cc-aabb-4534-a46a-0315a7572540'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:05.117453 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] The token cache is being invalidate because project b2d0730e64f647e6a53521fef94a887c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:05.118005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:05.118653 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6d5a84cc-aabb-4534-a46a-0315a7572540 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:05.120238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1440/2863] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:05 2022] DELETE /identity/v3/projects/b2d0730e64f647e6a53521fef94a887c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:05.125143 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.125617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.125719 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.143702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:05.145592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:05.208619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1424/2864] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:05.216939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.217294 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.217616 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.237678 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:05.239538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:05.248208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1441/2865] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:05.252893 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.262959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e06b552-beee-43b2-b3de-1bb5be24f7dd None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.267746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.268431 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.268713 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.268984 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.270256 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:07:05.270644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.272280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.283720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1425/2866] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:07:05.289140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.298722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c3afe-5f9d-4c52-956e-802e7079d3b3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.304041 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.304730 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.305017 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.305295 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.305997 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.307955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.339069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1442/2867] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:07:05.340553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.352022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03c21fa-4b57-458d-a1cb-186c1c0f132f admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.359785 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:05.412782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.413452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.413737 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.414013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.414701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.416704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.437228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1426/2868] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:05.442146 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.453307 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378e607a-067e-4f1c-8c61-291affc441b9 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.459252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.459918 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.460241 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.460525 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.461254 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.462863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.474487 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1443/2869] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:05.479499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.489621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee7dcb-a754-4e49-8fd3-e741819e43c9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.494604 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99176449-b98a-4dad-bb1f-4a82a9156615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.495257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99176449-b98a-4dad-bb1f-4a82a9156615 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.495544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99176449-b98a-4dad-bb1f-4a82a9156615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.495823 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99176449-b98a-4dad-bb1f-4a82a9156615 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.496595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99176449-b98a-4dad-bb1f-4a82a9156615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.498195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99176449-b98a-4dad-bb1f-4a82a9156615 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.509058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1427/2870] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:07:05.513256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.524566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab19a8-817b-4275-b582-d590a386735d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.529208 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.530111 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.530397 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.530676 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] PATH_INFO: `/v3/projects/dd5e4e1c38414803a0d828efe14d69ff/users/bd739fcb01284011ba618f1c5314313c/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.547892 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd5e4e1c38414803a0d828efe14d69ff, user_id=bd739fcb01284011ba618f1c5314313c, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.550880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.569671 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1444/2871] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:05 2022] PUT /identity/v3/projects/dd5e4e1c38414803a0d828efe14d69ff/users/bd739fcb01284011ba618f1c5314313c/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:05.582466 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.582754 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.583289 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.611728 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] MFA Rules not processed for user `bd739fcb01284011ba618f1c5314313c`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:05.613703 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:05.678659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1428/2872] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:07:05.683192 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.749728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d189d-5915-4576-9332-68ac86e8eab4 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.754615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.755278 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.755560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.755833 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.756573 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.758516 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.786167 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1445/2873] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:05.789862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.800849 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d0a4-8e83-480e-a3e8-97a89fea5f03 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.864111 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.865212 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.865504 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.865789 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.866505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.868819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.890716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1429/2874] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:07:05 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:05.896128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.906618 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ac14f2-fca6-4289-b475-826691f33c24 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.911954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.912658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.912944 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.913224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.913962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.915575 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.927384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1446/2875] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:05.931663 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.942575 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ba524-2bd0-4a67-8048-33f6d9a589ab admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:05.947204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:05.948075 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:05.948384 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:05.948661 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] PATH_INFO: `/v3/projects/4e03957d263240dabd0f75da90f473b0/users/73348edee6264a52a10aba019d93db81/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:05.951704 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:05.965062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e03957d263240dabd0f75da90f473b0, user_id=73348edee6264a52a10aba019d93db81, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:05.967852 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:05.980771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1430/2876] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:05 2022] PUT /identity/v3/projects/4e03957d263240dabd0f75da90f473b0/users/73348edee6264a52a10aba019d93db81/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:05.984947 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:05.996123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73ee8a-1aa6-4668-812c-cf04dfe23e01 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:06.020658 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:06.057475 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c6e46-eea8-4373-abf1-257a46e0f27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:06.058238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c6e46-eea8-4373-abf1-257a46e0f27f admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:06.058615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c6e46-eea8-4373-abf1-257a46e0f27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:06.059003 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c6e46-eea8-4373-abf1-257a46e0f27f admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:06.059926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c6e46-eea8-4373-abf1-257a46e0f27f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:06.061748 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643c6e46-eea8-4373-abf1-257a46e0f27f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:06.073033 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1447/2877] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:07:06.077339 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:06.087803 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5650da2-2080-4b5d-8c69-2adddb7453f6 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:06.092409 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:06.093282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:06.093559 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:06.093832 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] PATH_INFO: `/v3/projects/4e03957d263240dabd0f75da90f473b0/users/73348edee6264a52a10aba019d93db81/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:06.095800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e03957d263240dabd0f75da90f473b0, user_id=73348edee6264a52a10aba019d93db81, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:06.098492 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:06.108667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1431/2878] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:06 2022] PUT /identity/v3/projects/4e03957d263240dabd0f75da90f473b0/users/73348edee6264a52a10aba019d93db81/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:06.169506 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:06.169796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:06.170078 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:06.199810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] MFA Rules not processed for user `73348edee6264a52a10aba019d93db81`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:06.201048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:06.270767 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1448/2879] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:06.282332 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:06.284060 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:06.343068 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c674a70-70d8-448c-a851-29ef88dd92ba admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:06.346949 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:06.347892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:06.348366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:06.348657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:06.358176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:06.359622 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:06.370065 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1432/2880] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:06.444383 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:06.454042 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982248f-33d1-4d2a-906d-c5a28b68f2fa None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:06.458923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:06.459571 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:06.459855 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:06.460165 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:06.517710 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:06.520013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:06.529628 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1449/2881] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:13.095087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:13.096553 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:13.155196 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48bfbde-0a2d-4b36-93f7-4a1081c758a8 service cinder] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:13.158782 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:13.159432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:13.160452 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:13.160825 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:13.162619 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:13.228347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:13.229706 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:13.240092 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1433/2882] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:16.525309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=79 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:18.099391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:18.111199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4327c3-4884-47e0-9a58-33b2376cb2da service cinder] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:18.116442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:18.117283 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:18.117550 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:18.118127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:18.119987 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:18.182510 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:18.183981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:18.194396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1450/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:23.234904 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:24.648333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:24.650080 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:24.668336 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc439295-2185-480b-ae67-4e14c0363c2c service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:24.673475 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:24.674355 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:24.675207 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:24.675491 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] PATH_INFO: `/v3/users/24e3ac2f7c00476d8bcfb88aeb17293a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:24.676750 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e3ac2f7c00476d8bcfb88aeb17293a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:24.678730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:24.731267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e3ac2f7c00476d8bcfb88aeb17293a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:24.738461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e3ac2f7c00476d8bcfb88aeb17293a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:24.743600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e3ac2f7c00476d8bcfb88aeb17293a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:24.744241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e3ac2f7c00476d8bcfb88aeb17293a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:24.751323 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e3ac2f7c00476d8bcfb88aeb17293a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:24.769155 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1434/2884] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:24 2022] DELETE /identity/v3/users/24e3ac2f7c00476d8bcfb88aeb17293a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:24.941702 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:24.951625 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e43b6-ac03-4377-b7c2-502a22b65df7 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:25.009422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:25.010273 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:25.010552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:25.010826 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] PATH_INFO: `/v3/projects/2a62977afd3c4b17ac43429d7e593580` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:25.012002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a62977afd3c4b17ac43429d7e593580)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:25.013934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:25.061604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a62977afd3c4b17ac43429d7e593580', 'request_id': 'req-48bf7611-6bbd-4fe8-a515-0c5215c053bd'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.070048 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a62977afd3c4b17ac43429d7e593580', 'request_id': 'req-48bf7611-6bbd-4fe8-a515-0c5215c053bd'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.076287 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a62977afd3c4b17ac43429d7e593580', 'request_id': 'req-48bf7611-6bbd-4fe8-a515-0c5215c053bd'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.088178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] The token cache is being invalidate because project 2a62977afd3c4b17ac43429d7e593580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:25.088465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.089090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.090611 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1451/2885] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:24 2022] DELETE /identity/v3/projects/2a62977afd3c4b17ac43429d7e593580 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:25.094522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:25.095914 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:25.108768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61b01f-cd28-49ca-a349-91e9efe240e9 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:25.164747 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:25.165526 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:25.165807 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:25.166079 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] PATH_INFO: `/v3/users/052b9567cd724e3291561862ea503d52` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:25.167262 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=052b9567cd724e3291561862ea503d52)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:25.169224 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:25.214255 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052b9567cd724e3291561862ea503d52'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.221518 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052b9567cd724e3291561862ea503d52'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.223507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052b9567cd724e3291561862ea503d52'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.224152 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052b9567cd724e3291561862ea503d52'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.231399 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052b9567cd724e3291561862ea503d52'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.239226 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:25.251356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1435/2886] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:25 2022] DELETE /identity/v3/users/052b9567cd724e3291561862ea503d52 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:25.429408 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:25.431015 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:25.445106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7611-6bbd-4fe8-a515-0c5215c053bd admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:25.467158 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:25.506998 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:25.507865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:25.508184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:25.508460 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] PATH_INFO: `/v3/projects/e27ff170237048be9dc5a7a3bc58dd3b` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:25.509648 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e27ff170237048be9dc5a7a3bc58dd3b)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:25.511587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:25.556584 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e27ff170237048be9dc5a7a3bc58dd3b', 'request_id': 'req-a498036a-25bc-4b50-8476-7c7bd8faa14f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.563494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e27ff170237048be9dc5a7a3bc58dd3b', 'request_id': 'req-a498036a-25bc-4b50-8476-7c7bd8faa14f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.570238 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e27ff170237048be9dc5a7a3bc58dd3b', 'request_id': 'req-a498036a-25bc-4b50-8476-7c7bd8faa14f'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.580427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] The token cache is being invalidate because project e27ff170237048be9dc5a7a3bc58dd3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:25.580715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.581361 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-a498036a-25bc-4b50-8476-7c7bd8faa14f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:25.582883 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1452/2887] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:25 2022] DELETE /identity/v3/projects/e27ff170237048be9dc5a7a3bc58dd3b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:35.250829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:35.577413 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:53.643874 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:53.645360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:53.659522 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411bc4c-a7f9-4e42-8f84-c07832e6fb99 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:53.716878 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:53.717778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:53.718062 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:53.718334 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] PATH_INFO: `/v3/users/bd739fcb01284011ba618f1c5314313c` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:53.719545 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd739fcb01284011ba618f1c5314313c)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:53.721906 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:53.767911 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd739fcb01284011ba618f1c5314313c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:53.776945 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd739fcb01284011ba618f1c5314313c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:53.777591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd739fcb01284011ba618f1c5314313c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:53.778191 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd739fcb01284011ba618f1c5314313c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:53.785648 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd739fcb01284011ba618f1c5314313c'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:53.804240 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1436/2888] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:53 2022] DELETE /identity/v3/users/bd739fcb01284011ba618f1c5314313c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:53.808745 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436c771-269e-493d-822d-d7a58d555523 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:53.809185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436c771-269e-493d-822d-d7a58d555523 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:53.809511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436c771-269e-493d-822d-d7a58d555523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:53.827644 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-6436c771-269e-493d-822d-d7a58d555523 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:53.828865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6436c771-269e-493d-822d-d7a58d555523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:53.888310 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1453/2889] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:53.898000 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:53.899591 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:53.966901 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb2b523-869b-4df8-9b09-ebe1c7ac6cba admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:53.970580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:53.971222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:53.971498 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:53.971768 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:53.980812 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:53.982294 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:53.992521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1437/2890] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:54.257096 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436c771-269e-493d-822d-d7a58d555523 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:54.258766 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-6436c771-269e-493d-822d-d7a58d555523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:54.271597 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436c771-269e-493d-822d-d7a58d555523 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:54.276574 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:54.277444 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:54.277727 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:54.278002 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] PATH_INFO: `/v3/projects/dd5e4e1c38414803a0d828efe14d69ff` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:54.279212 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd5e4e1c38414803a0d828efe14d69ff)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:54.281195 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:54.329087 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd5e4e1c38414803a0d828efe14d69ff', 'request_id': 'req-1d9b9923-967d-4a89-9e25-1b407ef057cf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.337706 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd5e4e1c38414803a0d828efe14d69ff', 'request_id': 'req-1d9b9923-967d-4a89-9e25-1b407ef057cf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.344731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd5e4e1c38414803a0d828efe14d69ff', 'request_id': 'req-1d9b9923-967d-4a89-9e25-1b407ef057cf'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.356076 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] The token cache is being invalidate because project dd5e4e1c38414803a0d828efe14d69ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:54.356386 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.357071 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.358654 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1454/2891] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:54 2022] DELETE /identity/v3/projects/dd5e4e1c38414803a0d828efe14d69ff => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:54.362691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:54.373141 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df28cf-3f24-41fc-a791-1ae77227202a service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:54.381767 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:54.432360 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:54.433257 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:54.433541 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:54.433842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] PATH_INFO: `/v3/users/73348edee6264a52a10aba019d93db81` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:54.435011 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73348edee6264a52a10aba019d93db81)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:54.437009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:54.482085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73348edee6264a52a10aba019d93db81'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.489267 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73348edee6264a52a10aba019d93db81'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.490547 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73348edee6264a52a10aba019d93db81'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.491281 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73348edee6264a52a10aba019d93db81'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.498507 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-c252738e-1b02-4eb2-a575-74607538de03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73348edee6264a52a10aba019d93db81'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.516343 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1438/2892] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:54 2022] DELETE /identity/v3/users/73348edee6264a52a10aba019d93db81 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:54.791981 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:54.793461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:54.807219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9b9923-967d-4a89-9e25-1b407ef057cf admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:54.866785 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:54.867863 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:54.868183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:54.868465 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] PATH_INFO: `/v3/projects/4e03957d263240dabd0f75da90f473b0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:54.869666 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e03957d263240dabd0f75da90f473b0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:54.871602 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:54.904795 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:54.923742 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e03957d263240dabd0f75da90f473b0', 'request_id': 'req-4900ed6c-7120-4bff-9c62-ccf2499574a0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.931461 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e03957d263240dabd0f75da90f473b0', 'request_id': 'req-4900ed6c-7120-4bff-9c62-ccf2499574a0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.937786 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e03957d263240dabd0f75da90f473b0', 'request_id': 'req-4900ed6c-7120-4bff-9c62-ccf2499574a0'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.949227 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] The token cache is being invalidate because project 4e03957d263240dabd0f75da90f473b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:54.949517 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.950164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4900ed6c-7120-4bff-9c62-ccf2499574a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:54.951725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1455/2893] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:54 2022] DELETE /identity/v3/projects/4e03957d263240dabd0f75da90f473b0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:07:54.956631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:54.957093 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:54.957197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:54.976047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:54.977288 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:55.036393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1439/2894] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:55.045608 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.045901 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.046184 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.064013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:55.065230 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:55.073955 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1456/2895] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:55.078094 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.088614 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20d517-bce6-4eb5-a6b0-f9cd81cc6f74 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.093136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.093778 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.094058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.094331 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.095047 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:07:55.095347 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.096965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.108595 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1440/2896] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:07:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:07:55.112699 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.122309 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ee4f5-d734-4c8e-b88e-dbd02c4eda99 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.126957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.127606 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.127890 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.128198 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.128926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.131429 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.157930 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1457/2897] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:07:55.162415 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.172246 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528221c9-16b1-4f46-a712-7d8cd952513b admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.188714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:55.229366 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.230023 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.230306 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.230580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.231253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.233198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.253217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1441/2898] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:55.258129 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.268532 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267ff671-d7db-4d62-b283-44bb45048408 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.273275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.273934 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.274218 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.274494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.275224 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.276873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.288113 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1458/2899] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:55.292070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.301933 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571680a2-0d03-4b80-9a61-050ccdfa6fb0 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.307145 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c212851-aafe-4949-91b0-1fe655c4ad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.307804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c212851-aafe-4949-91b0-1fe655c4ad80 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.308110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c212851-aafe-4949-91b0-1fe655c4ad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.308401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c212851-aafe-4949-91b0-1fe655c4ad80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.309127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c212851-aafe-4949-91b0-1fe655c4ad80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.310752 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c212851-aafe-4949-91b0-1fe655c4ad80 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.320816 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1442/2900] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:55.324888 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.335312 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555b65f-188d-4e59-aa20-774c2d318da3 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.339847 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.340746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.341026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.341299 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] PATH_INFO: `/v3/projects/a0571371d85f400aa77f282a2bf50d29/users/6b465ccebbe34a658087ef92e91e0be3/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.356344 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0571371d85f400aa77f282a2bf50d29, user_id=6b465ccebbe34a658087ef92e91e0be3, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.359064 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.373024 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1459/2901] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:55 2022] PUT /identity/v3/projects/a0571371d85f400aa77f282a2bf50d29/users/6b465ccebbe34a658087ef92e91e0be3/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:55.384122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.384420 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.384701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.410413 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] MFA Rules not processed for user `6b465ccebbe34a658087ef92e91e0be3`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:55.411666 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:55.474965 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1443/2902] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:55.479464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.542068 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028276d-3603-4346-a0a0-7e3c52863a34 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.547304 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.547939 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.548256 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.548533 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.549211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.551119 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.578434 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1460/2903] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/projects => generated 389 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:55.582516 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.593148 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d9ba50-19de-4b58-87e8-6855a4c3aa5e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.651106 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.651696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.651980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.652290 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.652982 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.654897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.674204 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1444/2904] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/users => generated 301 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:07:55.678269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.688395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a78442-0d20-4937-9d43-13c9f10347db admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.692989 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.693643 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.693923 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.694199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.694915 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.697194 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.708275 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1461/2905] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:07:55.714070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.725574 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf59088-bfd6-4570-8e43-9880b6c50bb8 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.730371 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.731284 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.731565 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.731843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] PATH_INFO: `/v3/projects/cea4f54fbb9d40c4890ec7f3b4fb27cb/users/ae0270958dd543c6b17ba70bb422aa77/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.748237 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea4f54fbb9d40c4890ec7f3b4fb27cb, user_id=ae0270958dd543c6b17ba70bb422aa77, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.750931 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.753119 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:55.765718 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1445/2906] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:55 2022] PUT /identity/v3/projects/cea4f54fbb9d40c4890ec7f3b4fb27cb/users/ae0270958dd543c6b17ba70bb422aa77/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:55.769762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.783586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c7d828-2313-4bfa-81c1-cc56853e484c admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.803214 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:55.841595 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae0032-795c-4605-9ddb-b6e4d57f6cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.842258 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae0032-795c-4605-9ddb-b6e4d57f6cff admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.842541 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae0032-795c-4605-9ddb-b6e4d57f6cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.842815 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae0032-795c-4605-9ddb-b6e4d57f6cff admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.843568 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae0032-795c-4605-9ddb-b6e4d57f6cff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.845197 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ae0032-795c-4605-9ddb-b6e4d57f6cff admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.854927 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1462/2907] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:55.859203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:55.869967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14c75-f2ff-49b0-a3f7-328fc5b5139c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:55.874657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:55.875612 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.875892 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.876201 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] PATH_INFO: `/v3/projects/cea4f54fbb9d40c4890ec7f3b4fb27cb/users/ae0270958dd543c6b17ba70bb422aa77/roles/56597e05853f4660b09a96c651dd0c7a` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.878186 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cea4f54fbb9d40c4890ec7f3b4fb27cb, user_id=ae0270958dd543c6b17ba70bb422aa77, role_id=56597e05853f4660b09a96c651dd0c7a)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:55.881514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:55.891996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1446/2908] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:55 2022] PUT /identity/v3/projects/cea4f54fbb9d40c4890ec7f3b4fb27cb/users/ae0270958dd543c6b17ba70bb422aa77/roles/56597e05853f4660b09a96c651dd0c7a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:55.902637 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:55.903116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:55.903116 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:55.928817 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] MFA Rules not processed for user `ae0270958dd543c6b17ba70bb422aa77`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:55.930026 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:55.999609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1463/2909] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:56.010787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:56.012509 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:56.077784 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4d72-17fa-4f5b-a474-4002019603cc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:56.081406 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a550e36-5d00-402d-b2c5-7ad2b00fb17f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:56.082041 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a550e36-5d00-402d-b2c5-7ad2b00fb17f service cinder] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:56.082321 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a550e36-5d00-402d-b2c5-7ad2b00fb17f service cinder] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:56.082592 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a550e36-5d00-402d-b2c5-7ad2b00fb17f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:56.092181 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a550e36-5d00-402d-b2c5-7ad2b00fb17f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:56.093608 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a550e36-5d00-402d-b2c5-7ad2b00fb17f service cinder] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:56.103091 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1447/2910] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:56.232700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:56.294027 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a768889-25ab-466b-9f14-0f388f43023b None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:56.298464 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:56.299323 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:56.299601 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:56.299873 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] PATH_INFO: `/v3/users/6b465ccebbe34a658087ef92e91e0be3` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:56.301077 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b465ccebbe34a658087ef92e91e0be3)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:56.303587 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:56.347391 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b465ccebbe34a658087ef92e91e0be3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.349105 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:56.355538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b465ccebbe34a658087ef92e91e0be3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.356511 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b465ccebbe34a658087ef92e91e0be3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.357123 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b465ccebbe34a658087ef92e91e0be3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.369743 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b465ccebbe34a658087ef92e91e0be3'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.382760 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1464/2911] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:56 2022] DELETE /identity/v3/users/6b465ccebbe34a658087ef92e91e0be3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:56.387401 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:56.387690 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:56.387967 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:56.406241 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:56.407424 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:56.464758 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1448/2912] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:56.474867 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:56.476539 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:56.532291 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e60e1-a733-42ef-a708-51e5bbbc00ee admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:56.535882 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:56.537102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:56.537534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:56.537810 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:56.546704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:56.548170 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:56.558098 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1465/2913] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:07:56.839074 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:56.840638 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:56.863225 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df02a18-86ad-430e-a3c9-1fac972bbb7e None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:56.869586 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:56.870429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:56.870740 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:56.871040 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] PATH_INFO: `/v3/projects/a0571371d85f400aa77f282a2bf50d29` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:56.872314 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0571371d85f400aa77f282a2bf50d29)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:56.874253 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:56.926903 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0571371d85f400aa77f282a2bf50d29', 'request_id': 'req-4582fc1e-3dfc-4b35-8612-53c957fec403'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.933681 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0571371d85f400aa77f282a2bf50d29', 'request_id': 'req-4582fc1e-3dfc-4b35-8612-53c957fec403'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.939388 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0571371d85f400aa77f282a2bf50d29', 'request_id': 'req-4582fc1e-3dfc-4b35-8612-53c957fec403'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.948499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:56.953726 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] The token cache is being invalidate because project a0571371d85f400aa77f282a2bf50d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:56.954054 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.955154 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:56.957012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1449/2914] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:56 2022] DELETE /identity/v3/projects/a0571371d85f400aa77f282a2bf50d29 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:56.963056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:56.973396 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775cfc51-6121-4e95-8592-c4594df7e446 service neutron] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:57.030566 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:57.031352 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:57.031636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:57.031913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] PATH_INFO: `/v3/users/ae0270958dd543c6b17ba70bb422aa77` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:57.033139 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae0270958dd543c6b17ba70bb422aa77)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:57.035074 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:57.080211 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae0270958dd543c6b17ba70bb422aa77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.088469 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae0270958dd543c6b17ba70bb422aa77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.089109 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae0270958dd543c6b17ba70bb422aa77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.089782 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae0270958dd543c6b17ba70bb422aa77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.102438 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:57.103734 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-624eb55a-b374-4006-bd4e-40061d0b81ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae0270958dd543c6b17ba70bb422aa77'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.118536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1466/2915] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:07:56 2022] DELETE /identity/v3/users/ae0270958dd543c6b17ba70bb422aa77 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:57.642104 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:57.643656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:57.657811 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4582fc1e-3dfc-4b35-8612-53c957fec403 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:57.717110 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:57.717966 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:57.718252 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:57.718530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] PATH_INFO: `/v3/projects/cea4f54fbb9d40c4890ec7f3b4fb27cb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:57.719714 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cea4f54fbb9d40c4890ec7f3b4fb27cb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:57.721721 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:57.769664 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cea4f54fbb9d40c4890ec7f3b4fb27cb', 'request_id': 'req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.776260 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cea4f54fbb9d40c4890ec7f3b4fb27cb', 'request_id': 'req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.781730 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cea4f54fbb9d40c4890ec7f3b4fb27cb', 'request_id': 'req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.794173 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] The token cache is being invalidate because project cea4f54fbb9d40c4890ec7f3b4fb27cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80200) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:07:57.794463 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.795085 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-b372599a-91ab-4a6a-bab1-ca3c878e6d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:07:57.796657 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1450/2916] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:07:57 2022] DELETE /identity/v3/projects/cea4f54fbb9d40c4890ec7f3b4fb27cb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:57.801503 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:57.801796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:57.802620 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:57.821202 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:57.822398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:57.880134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1467/2917] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:57.889479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:57.889771 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:57.890048 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:57.908203 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:57.909461 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:57.917368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1451/2918] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:57.921635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:57.933583 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c4702-6580-4420-a6d5-be7971d8eaa9 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:57.938832 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:57.939497 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:57.939804 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:57.940112 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] PATH_INFO: `/v3/domains` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:57.940852 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80199) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:07:57.941159 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:57.943489 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:57.955411 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1468/2919] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:07:57.959988 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:57.969519 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad0e1a-10a3-406f-93da-899eedd08cb0 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:57.975017 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:57.975842 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:57.976153 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:57.976429 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] PATH_INFO: `/v3/projects` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:57.977122 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:57.979012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:58.012471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1452/2920] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:07:57 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:58.016731 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:58.027177 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9467a705-90bc-42f2-b8a1-daf6e684c629 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:58.086384 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:58.087061 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:58.087343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:58.087615 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] PATH_INFO: `/v3/users` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:58.088364 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:58.090320 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:58.113253 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1469/2921] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:07:58 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:07:58.118337 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:58.128840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d8b36-b1f6-4915-b9db-3448cd45398c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:58.134039 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:58.134636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:58.134923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:58.135197 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:58.135926 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:58.137542 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:58.149028 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1453/2922] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 15 20:07:58.153128 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:58.163398 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd4135-5772-42bb-9a89-35cd4fd9dd8d admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:58.168271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a63f7e1-c5ca-4124-b712-9f854a3e1397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:58.168935 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63f7e1-c5ca-4124-b712-9f854a3e1397 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:58.169292 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63f7e1-c5ca-4124-b712-9f854a3e1397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:58.169604 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a63f7e1-c5ca-4124-b712-9f854a3e1397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:58.170404 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63f7e1-c5ca-4124-b712-9f854a3e1397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:58.172069 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a63f7e1-c5ca-4124-b712-9f854a3e1397 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:58.183286 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1470/2923] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:07:58.187198 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:58.198053 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c2ca66-655c-49f9-bed9-b48f5cebff0a admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:58.202821 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:58.203707 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:58.203983 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:58.204283 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] PATH_INFO: `/v3/projects/268a89c251254fb3a53d2a8b18a13b10/users/c4e21c286aff41d992b75541622d5738/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:58.220818 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268a89c251254fb3a53d2a8b18a13b10, user_id=c4e21c286aff41d992b75541622d5738, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:58.223490 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:58.237217 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1454/2924] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:07:58 2022] PUT /identity/v3/projects/268a89c251254fb3a53d2a8b18a13b10/users/c4e21c286aff41d992b75541622d5738/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:07:58.247798 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:58.248106 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:58.248395 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:58.275709 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] MFA Rules not processed for user `c4e21c286aff41d992b75541622d5738`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:07:58.276979 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:58.319178 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:58.342690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1471/2925] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:07:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:07:58.354558 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:07:58.356309 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:07:58.385701 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:07:58.416009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81425cc6-a000-46de-bae7-db328332dd26 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:07:58.420373 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6802452aab2f43d4b965be689758b611', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:07:58.421016 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:07:58.421302 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:07:58.421580 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:07:58.430499 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:07:58.431943 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:07:58.440677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1455/2926] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:07:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:08.339715 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=8 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:08.436741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=15 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:27.805467 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:27.806926 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:27.879586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa8bcd-13e9-43db-bb33-3a97658f03f3 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:27.885161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37f8217df17a44a989ce1c7ab8a9ac1a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:27.885803 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:27.886085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:27.886359 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:27.888164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:27.949700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:27.951330 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:27.961047 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1472/2927] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:08:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:35.667927 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:35.669417 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:35.727562 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a2c03-0159-446a-be05-2980ffb7afe8 service glance] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:35.731136 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:35.731783 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:35.732061 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:35.732368 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:35.734123 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:35.802149 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:35.803338 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:35.813897 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1456/2928] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:37.956380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=56 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:39.056591 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:39.058188 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:39.171000 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f1afce-f1bf-4e89-af01-dc0f8797d1cf service nova] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:39.176175 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:39.176837 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:39.177161 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:39.177427 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:39.179494 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:39.191728 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:39.194247 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:39.205007 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1473/2929] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:08:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:42.630353 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:42.631820 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:42.645022 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fbb99c-2596-482b-ab93-6b7d5bfecd83 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:42.650570 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:42.651441 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:42.651723 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:42.652003 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] PATH_INFO: `/v3/users/0699a27504da472e96b2dbd52d6cf301` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:42.653222 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0699a27504da472e96b2dbd52d6cf301)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:42.655151 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:42.703087 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0699a27504da472e96b2dbd52d6cf301'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:42.713734 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0699a27504da472e96b2dbd52d6cf301'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:42.714482 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0699a27504da472e96b2dbd52d6cf301'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:42.715423 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0699a27504da472e96b2dbd52d6cf301'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:42.723631 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0699a27504da472e96b2dbd52d6cf301'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:42.742274 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1457/2930] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:08:42 2022] DELETE /identity/v3/users/0699a27504da472e96b2dbd52d6cf301 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:42.941670 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:42.953864 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac16ca7-7875-4346-975b-869807d72d81 service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:43.028414 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:43.030073 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:43.030445 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:43.030726 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] PATH_INFO: `/v3/projects/bc71f6e4fba640f2b41e5b3e4eb7f14d` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:43.031940 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc71f6e4fba640f2b41e5b3e4eb7f14d)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:43.033898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:43.085769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc71f6e4fba640f2b41e5b3e4eb7f14d', 'request_id': 'req-4b18a3de-625f-4ea9-9167-420de3539a8b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.093884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc71f6e4fba640f2b41e5b3e4eb7f14d', 'request_id': 'req-4b18a3de-625f-4ea9-9167-420de3539a8b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.101722 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc71f6e4fba640f2b41e5b3e4eb7f14d', 'request_id': 'req-4b18a3de-625f-4ea9-9167-420de3539a8b'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.113534 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] The token cache is being invalidate because project bc71f6e4fba640f2b41e5b3e4eb7f14d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:08:43.113821 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.114482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.116182 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1474/2931] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:08:42 2022] DELETE /identity/v3/projects/bc71f6e4fba640f2b41e5b3e4eb7f14d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:43.120505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:43.122067 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:43.138587 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb53f2-0d03-42db-a9cb-885e0c49ae14 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:43.163635 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:43.205097 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:43.205884 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:43.206228 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:43.206528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] PATH_INFO: `/v3/users/dd4ddc0930924161b67a5d428a327edb` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:43.207909 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4ddc0930924161b67a5d428a327edb)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:43.210176 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:43.257118 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4ddc0930924161b67a5d428a327edb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.264780 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4ddc0930924161b67a5d428a327edb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.265398 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4ddc0930924161b67a5d428a327edb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.266030 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4ddc0930924161b67a5d428a327edb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.273333 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4ddc0930924161b67a5d428a327edb'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.290695 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1458/2932] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:08:43 2022] DELETE /identity/v3/users/dd4ddc0930924161b67a5d428a327edb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 15 20:08:43.472700 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:43.474688 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:43.488865 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18a3de-625f-4ea9-9167-420de3539a8b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:43.495538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:43.573993 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:43.574871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:43.575148 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:43.575609 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] PATH_INFO: `/v3/projects/a2a6385911fa4d568b797dc90f3020b9` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:43.576656 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a6385911fa4d568b797dc90f3020b9)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:43.578613 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:43.634271 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a6385911fa4d568b797dc90f3020b9', 'request_id': 'req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.643063 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a6385911fa4d568b797dc90f3020b9', 'request_id': 'req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.651166 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a6385911fa4d568b797dc90f3020b9', 'request_id': 'req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.665422 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] The token cache is being invalidate because project a2a6385911fa4d568b797dc90f3020b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:08:43.665773 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.666402 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-6181a2ac-4ad4-4fd7-8a5a-5529374d730c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:43.667975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1475/2933] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:08:43 2022] DELETE /identity/v3/projects/a2a6385911fa4d568b797dc90f3020b9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:47.583969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:47.585540 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:47.598358 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e7bbf-b9bd-49bf-b2f5-b4a4a0b6b056 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:47.662687 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:47.663432 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:47.663716 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:47.663990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] PATH_INFO: `/v3/users/c4e21c286aff41d992b75541622d5738` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:47.665282 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e21c286aff41d992b75541622d5738)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:47.667219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:47.714105 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:47.717403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e21c286aff41d992b75541622d5738'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:47.727127 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e21c286aff41d992b75541622d5738'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:47.727756 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e21c286aff41d992b75541622d5738'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:47.728434 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e21c286aff41d992b75541622d5738'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:47.735793 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e21c286aff41d992b75541622d5738'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:47.754261 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1459/2934] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:08:47 2022] DELETE /identity/v3/users/c4e21c286aff41d992b75541622d5738 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:47.759544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:47.760060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:47.760226 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:47.778809 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:08:47.780012 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:47.839085 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1476/2935] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:08:47.848796 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:47.850802 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:47.907449 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14aea8a-533a-44dc-86b1-f183e6610675 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:47.911135 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd1d42-91e7-49c6-9133-2127f024e60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:47.911815 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1d42-91e7-49c6-9133-2127f024e60e service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:47.912829 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1d42-91e7-49c6-9133-2127f024e60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:47.913164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd1d42-91e7-49c6-9133-2127f024e60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:47.922539 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1d42-91e7-49c6-9133-2127f024e60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:47.923980 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd1d42-91e7-49c6-9133-2127f024e60e service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:47.933819 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1460/2936] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:08:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:48.209846 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.211343 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:48.227060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13411924-7295-4fb7-a898-a0380cf79b2e None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.231690 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.232635 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.232911 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.233187 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] PATH_INFO: `/v3/projects/268a89c251254fb3a53d2a8b18a13b10` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.234409 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268a89c251254fb3a53d2a8b18a13b10)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.236373 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.276219 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:48.288200 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268a89c251254fb3a53d2a8b18a13b10', 'request_id': 'req-b303d8c3-5940-4009-98ab-d174ea584356'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:48.295673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268a89c251254fb3a53d2a8b18a13b10', 'request_id': 'req-b303d8c3-5940-4009-98ab-d174ea584356'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:48.301949 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268a89c251254fb3a53d2a8b18a13b10', 'request_id': 'req-b303d8c3-5940-4009-98ab-d174ea584356'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:48.312342 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] The token cache is being invalidate because project 268a89c251254fb3a53d2a8b18a13b10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:08:48.312629 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:48.313269 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-b303d8c3-5940-4009-98ab-d174ea584356 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:48.314796 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1477/2937] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:08:48 2022] DELETE /identity/v3/projects/268a89c251254fb3a53d2a8b18a13b10 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:48.320652 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.321012 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.321296 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.339326 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:08:48.340629 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:48.399677 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1461/2938] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 15 20:08:48.408428 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.408723 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.409005 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.428172 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:08:48.429390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:48.437351 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1478/2939] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:08:48.441696 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.452278 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d48d9e2-8a2e-4d3a-8621-ae793b3cb3df None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.457056 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.457720 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.458037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.458318 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.459700 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80200) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 15 20:08:48.460013 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.461668 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.473471 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1462/2940] 10.222.0.17 () {58 vars in 1152 bytes} [Fri Jul 15 20:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 15 20:08:48.478140 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.487433 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7e7e6-bce2-4417-a6f4-b325cc843b98 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.493049 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.493705 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.494252 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.494524 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] PATH_INFO: `/v3/projects` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.495210 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.497257 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.523586 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1479/2941] 10.222.0.17 () {60 vars in 1152 bytes} [Fri Jul 15 20:08:48 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 15 20:08:48.527659 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.538902 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76330f54-c418-4878-91e7-372b02350b21 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.586248 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:48.597656 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.598266 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.598551 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.598824 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] PATH_INFO: `/v3/users` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.599567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.601521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.621746 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1463/2942] 10.222.0.17 () {60 vars in 1143 bytes} [Fri Jul 15 20:08:48 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 15 20:08:48.625871 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.637134 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028e0f3-976b-4f9f-b1b2-74a8b6293209 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.641985 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.642634 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.642913 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.643185 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.643916 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.645560 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.657248 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1480/2943] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:08:48.661670 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.672804 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f857d3-a9ed-4f5a-98db-8ec38e9af958 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.678403 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02bf41-ba71-47d7-a9ad-7aede650cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.679070 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02bf41-ba71-47d7-a9ad-7aede650cc16 admin admin] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.679473 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02bf41-ba71-47d7-a9ad-7aede650cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.679757 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02bf41-ba71-47d7-a9ad-7aede650cc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.680528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02bf41-ba71-47d7-a9ad-7aede650cc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.682169 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02bf41-ba71-47d7-a9ad-7aede650cc16 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.693015 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1464/2944] 10.222.0.17 () {58 vars in 1121 bytes} [Fri Jul 15 20:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 15 20:08:48.697829 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.708010 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067350e1-2e42-49ad-b66b-2df3b5c9d721 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.713013 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.713903 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.714183 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.714458 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] PATH_INFO: `/v3/projects/98fa31e6349e429aaf99d935860c5fe6/users/52601f5168b74c6285c5a70efd54e334/roles/8dbe145322294429a2b040dd40e9a8e0` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.730523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=98fa31e6349e429aaf99d935860c5fe6, user_id=52601f5168b74c6285c5a70efd54e334, role_id=8dbe145322294429a2b040dd40e9a8e0)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.733279 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.750367 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1481/2945] 10.222.0.17 () {60 vars in 1482 bytes} [Fri Jul 15 20:08:48 2022] PUT /identity/v3/projects/98fa31e6349e429aaf99d935860c5fe6/users/52601f5168b74c6285c5a70efd54e334/roles/8dbe145322294429a2b040dd40e9a8e0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:48.763521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] REQUEST_METHOD: `POST` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.763826 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.764164 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.793010 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.auth.core [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] MFA Rules not processed for user `52601f5168b74c6285c5a70efd54e334`. Rule list: `[]` (Enabled: `True`). {{(pid=80200) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:08:48.794229 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:48.869521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1465/2946] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:08:48.881072 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:48.882789 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:48.961260 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7aac5e-bc73-45f5-8357-0ff499cb1c4b admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:48.965016 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07ea326-e99a-40d3-b8b1-2cc05a1dbcba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43fda9627eb84093bb59ceb1a553baf7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:48.965675 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea326-e99a-40d3-b8b1-2cc05a1dbcba service cinder] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:48.965962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea326-e99a-40d3-b8b1-2cc05a1dbcba service cinder] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:48.966244 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07ea326-e99a-40d3-b8b1-2cc05a1dbcba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:48.977264 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ea326-e99a-40d3-b8b1-2cc05a1dbcba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:48.978735 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07ea326-e99a-40d3-b8b1-2cc05a1dbcba service cinder] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:48.988272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1482/2947] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:08:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:51.529627 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:51.594589 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1c716-2909-49a3-ac89-8f147f805622 None None] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:51.599600 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:51.600549 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:51.600836 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:51.601139 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] PATH_INFO: `/v3/users/52601f5168b74c6285c5a70efd54e334` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:51.602505 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52601f5168b74c6285c5a70efd54e334)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:51.604521 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:51.651453 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52601f5168b74c6285c5a70efd54e334'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:51.658647 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52601f5168b74c6285c5a70efd54e334'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:51.659280 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52601f5168b74c6285c5a70efd54e334'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:51.659882 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52601f5168b74c6285c5a70efd54e334'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:51.665953 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:51.667923 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52601f5168b74c6285c5a70efd54e334'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:51.687137 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1466/2948] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:08:51 2022] DELETE /identity/v3/users/52601f5168b74c6285c5a70efd54e334 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:08:51.691877 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:51.692390 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:51.692485 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:51.711423 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] MFA Rules not processed for user `477b9e36f0344a75a2c77704966bc3a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:08:51.712645 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:51.759962 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:08:51.777101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1483/2949] 10.222.0.17 () {58 vars in 957 bytes} [Fri Jul 15 20:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:08:51.786840 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:51.788905 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:51.850617 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e7fb3-3586-450f-b05c-03d2ac7edf3c admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:51.854349 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:51.855007 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:51.855291 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:51.855567 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:51.864887 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:51.866532 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:51.876378 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1467/2950] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:08:52.175060 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:08:52.176714 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:08:52.189523 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7018c1-16c8-4f28-9a0a-69d9160a1763 None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:08:52.194980 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:08:52.195856 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:08:52.196164 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:08:52.196442 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] PATH_INFO: `/v3/projects/98fa31e6349e429aaf99d935860c5fe6` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:08:52.197630 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98fa31e6349e429aaf99d935860c5fe6)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:08:52.199552 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:08:52.248118 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98fa31e6349e429aaf99d935860c5fe6', 'request_id': 'req-82658d12-0c1c-4aff-ba73-49b4c80d0583'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:52.255393 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98fa31e6349e429aaf99d935860c5fe6', 'request_id': 'req-82658d12-0c1c-4aff-ba73-49b4c80d0583'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:52.262416 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98fa31e6349e429aaf99d935860c5fe6', 'request_id': 'req-82658d12-0c1c-4aff-ba73-49b4c80d0583'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:52.273019 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] The token cache is being invalidate because project 98fa31e6349e429aaf99d935860c5fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:08:52.273316 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:52.274056 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:08:52.275988 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1484/2951] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:08:52 2022] DELETE /identity/v3/projects/98fa31e6349e429aaf99d935860c5fe6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:09:01.872077 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=34,keystone:DELETE=2 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:09:02.269725 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:09:35.787528 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:09:35.789009 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:09:35.865800 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9117c601-ecae-45f1-ab84-8daea04c2ef3 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:09:35.931247 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:09:35.932195 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:09:35.932479 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:09:35.932860 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:09:35.934667 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:09:35.946493 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:09:35.948738 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:09:35.959330 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1468/2952] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:09:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 15 20:09:45.954749 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=58 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:10:03.381673 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:03.383101 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:03.442143 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82658d12-0c1c-4aff-ba73-49b4c80d0583 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:03.445680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:03.446327 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:03.446612 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:03.446891 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:03.448769 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:03.544640 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:03.546090 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5c18f-134f-4bc6-9b82-3d748ee100af service neutron] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:03.565966 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1485/2953] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:10:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:10:04.743113 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:04.744869 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:04.760058 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fe660-4991-4539-87b8-af56ed865b21 service placement] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:04.765131 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f135ce75fbc4b12977cc8f009083ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:04.765935 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:04.766286 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:04.766621 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:04.768787 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:04.785969 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:04.787881 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:04.802190 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1469/2954] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:10:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:10:07.223243 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] REQUEST_METHOD: `POST` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:07.223704 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:07.223880 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:07.242746 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.auth.core [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] MFA Rules not processed for user `62f4fa05f65c4643898254268882b8f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80199) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 15 20:10:07.243950 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:07.302441 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1486/2955] 10.222.0.17 () {58 vars in 967 bytes} [Fri Jul 15 20:10:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 15 20:10:07.344044 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:07.345571 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:07.417854 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0347f7-8853-4662-919b-6b52ad0529a8 service neutron] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:07.423640 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b3c45809bac4828bcde30494ab7a0d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:07.424351 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:07.424660 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:07.424959 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:07.433476 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:07.435158 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:07.445393 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1470/2956] 10.222.0.17 () {60 vars in 1384 bytes} [Fri Jul 15 20:10:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 15 20:10:14.430018 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:14.431536 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:14.447884 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8595c67d-3124-405b-bf8e-5411f4cbcc4d None None] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:14.454762 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:14.455650 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] REQUEST_METHOD: `GET` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:14.455954 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:14.456328 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:14.458347 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:14.472894 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:14.475366 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:14.487623 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1487/2957] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:10:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:10:14.502114 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:14.503662 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:14.610702 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197dbb8-40f0-46c4-8785-07267fa82033 service swift] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:14.616038 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:14.617175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:14.617514 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:14.617996 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] PATH_INFO: `/v3/users/afd610c316f644b68fb1b61824384f01` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:14.619569 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afd610c316f644b68fb1b61824384f01)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:14.622219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:14.676644 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd610c316f644b68fb1b61824384f01'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:14.684120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:10:14.692341 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd610c316f644b68fb1b61824384f01'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:14.693120 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd610c316f644b68fb1b61824384f01'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:14.693862 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd610c316f644b68fb1b61824384f01'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:14.702674 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd610c316f644b68fb1b61824384f01'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:14.721037 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1471/2958] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:10:14 2022] DELETE /identity/v3/users/afd610c316f644b68fb1b61824384f01 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:10:14.925957 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:14.939772 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6de11-c1b2-4c52-8223-604dd46549bd service placement] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:14.944043 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=100 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:10:15.002482 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:15.003346 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:15.003624 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:15.003899 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] PATH_INFO: `/v3/projects/c128406f10d844baa248e415f8169026` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:15.005127 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c128406f10d844baa248e415f8169026)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:15.007006 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:15.053741 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c128406f10d844baa248e415f8169026', 'request_id': 'req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.060898 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c128406f10d844baa248e415f8169026', 'request_id': 'req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.068199 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c128406f10d844baa248e415f8169026', 'request_id': 'req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.078680 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] The token cache is being invalidate because project c128406f10d844baa248e415f8169026 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:10:15.078975 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.079636 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.081216 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1488/2959] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:10:14 2022] DELETE /identity/v3/projects/c128406f10d844baa248e415f8169026 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:10:15.085165 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:15.086560 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:15.099474 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51814bc5-c5b0-4671-a5f3-d2f7c793e798 admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:15.157544 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:15.158412 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:15.158697 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:15.158977 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] PATH_INFO: `/v3/users/752da97b9da945459a09a51b626f73f2` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:15.160219 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=752da97b9da945459a09a51b626f73f2)` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:15.162175 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:15.208843 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752da97b9da945459a09a51b626f73f2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.216751 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752da97b9da945459a09a51b626f73f2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.217411 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752da97b9da945459a09a51b626f73f2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.218900 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752da97b9da945459a09a51b626f73f2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.228761 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.notifications [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752da97b9da945459a09a51b626f73f2'} {{(pid=80200) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.249356 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1472/2960] 10.222.0.17 () {58 vars in 1223 bytes} [Fri Jul 15 20:10:15 2022] DELETE /identity/v3/users/752da97b9da945459a09a51b626f73f2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:10:15.423221 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Authenticating user token {{(pid=80199) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:10:15.424759 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.fernet_utils [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80199) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:10:15.438538 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20cdd65-08dd-4eab-9ca1-7ab8ab092265 admin admin] Validating token access rules against request {{(pid=80199) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:10:15.497037 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '477b9e36f0344a75a2c77704966bc3a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '98ca6d3d93034fc49fb3c9265ccf0dcb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80199) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:10:15.497820 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:10:15.498102 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:10:15.498380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] PATH_INFO: `/v3/projects/0362a5b1fb294744bd25f50c3c0e5a67` {{(pid=80199) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:10:15.500420 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0362a5b1fb294744bd25f50c3c0e5a67)` {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:10:15.502649 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] RBAC: Authorization granted {{(pid=80199) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:10:15.543964 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:10:15.554190 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0362a5b1fb294744bd25f50c3c0e5a67', 'request_id': 'req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.563272 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0362a5b1fb294744bd25f50c3c0e5a67', 'request_id': 'req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.570544 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0362a5b1fb294744bd25f50c3c0e5a67', 'request_id': 'req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66'} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.582889 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] The token cache is being invalidate because project 0362a5b1fb294744bd25f50c3c0e5a67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80199) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 15 20:10:15.583284 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.583963 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG keystone.notifications [None req-7dd018c4-4c01-4d8b-b4b3-b5db7c647c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80199) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 15 20:10:15.585812 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: [pid: 80199|app: 0|req: 1489/2961] 10.222.0.17 () {58 vars in 1232 bytes} [Fri Jul 15 20:10:15 2022] DELETE /identity/v3/projects/0362a5b1fb294744bd25f50c3c0e5a67 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 15 20:10:25.244968 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:DELETE=13,keystone:SELECT=65,keystone:INSERT=1 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:10:25.579380 dv-cinder-849690-2-iscsi devstack@keystone.service[80199]: DEBUG dbcounter [-] [80199] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80199) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 15 20:14:39.748636 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Authenticating user token {{(pid=80200) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 15 20:14:39.750090 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:14:39.853776 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef4dccc-5daa-49b3-b26f-cc27144e50fc admin admin] Validating token access rules against request {{(pid=80200) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 15 20:14:39.913530 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '350d4be3651647829ae3f7e903326891', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd476a68e6b254166ae1df0ccdc93e83d', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80200) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 15 20:14:39.914194 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] REQUEST_METHOD: `GET` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 15 20:14:39.914465 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] SCRIPT_NAME: `/identity` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 15 20:14:39.914741 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80200) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 15 20:14:39.916524 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.fernet_utils [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80200) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 15 20:14:39.928407 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 15 20:14:39.929991 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36158ff7-bc2e-46ad-898a-b25206dc68ce service placement] RBAC: Authorization granted {{(pid=80200) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 15 20:14:39.939990 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: [pid: 80200|app: 0|req: 1473/2962] 10.222.0.17 () {60 vars in 1365 bytes} [Fri Jul 15 20:14:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 15 20:14:49.935691 dv-cinder-849690-2-iscsi devstack@keystone.service[80200]: DEBUG dbcounter [-] [80200] Writing DB stats keystone:SELECT=69 {{(pid=80200) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}